[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant] Lanham, Reinforcement Learning, from Games to Geologic Interpretation. htt ps://medium.com/@cxbxmxcx/reinforcement-learning-from-games-to-geologi c-interpretation-93757664f0e4, 2019 (accessed 2 October 2020). [18] M. Lanham, Reinforcement Learning, It’s Coming not Just for Games. https://me dium.com/@cxbxmxcx/reinforcement-learning-its-coming-not-just-for-games-6b6 064e02bbe, 2019. [19] D. Schwung, A. Schwung, S. Ding, Actor-critic reinforcement learning for energy optimization in hybrid production environments, Int. J. Comput. 18 (2019) 360–371. [20] I. Carlucho, M. de Paula, S. Wang, Y. Petillot, G.G. Acosta, Adaptive low-level control of autonomous underwater vehicles using deep reinforcement learning, Robot. Auton. Syst. 107 (2018) 71–86, https://doi.org/10.1016/j. robot.2018.05.016. [21] B. Maidl, M. Thewes, U. Maidl, D. Sturge, Handbook of Tunnel Engineering, First, Engl. ed., Ernst/Wiley, Berlin, 2013. [22] W. Stipek, R. Galler, M. Bauer (Eds.), 50 Years of NATM: Experience Reports, ITA, Austria, Wien, 2012. [23] Norwegian Tunnelling Society (Ed.), Norwegian Tunnelling Technology: Publication no. 23, 2014. [24] S. Pelizza, D. Peila, Soil and rock reinforcements in tunnelling, Tunn. Undergr. Space Technol. 8 (1993) 357–372, https://doi.org/10.1016/0886-7798(93)90020- V. [25] P. Lunardi, Design and Construction of Tunnels: Analysis of Controlled Deformation in Rocks and Soils (ADECO-RS), Springer-Verlag, Berlin, Heidelberg, 2008. [26] S. Raschka, V. Mirjalili, Python Machine Learning - Third Edition: Machine learning and deep learning with python, scikit … -learn, and tensorflow 2, Packt Publishing Limited, [S.l.], 2019. [27] R.S. Sutton, A.G. Barto, Reinforcement Learning: An Introduction, Second edition, The MIT Press, Cambridge Massachusetts, 2018. [28] ITA Working Group Conventional Tunnelling, General Report on Conventional Tunnelling Method, Distrib, ITA Secretariat c/o EPFL, Lausanne, 2009. [29] P.A. Vermeer, N. Ruse, T. Marcher, Tunnel heading stability in drained ground, Felsbau 20 (2002) 8–18. [30] M.A. Kanji, Critical issues in soft rocks, J. Rock Mech. Geotech. Eng. 6 (2014) 186–195, https://doi.org/10.1016/j.jrmge.2014.04.002. [31] T. Marcher, S. Stauder, M. Winkler, HSSR – Ein Versuch der Einordnung und Abgrenzung des Materials, in: T. Marcher (Ed.), Beitr¨age zum 1. Hard Soil – Soft Rock (HSSR) Minisymposium: Charakterisierung, Modellierung und experimentelle Untersuchungen von Uebergangsgesteinen – Aktuelles aus Forschung und Entwicklung, Graz, 2020. [32] G. Anagnostou, K. Kov´ari, Face stability conditions with earth-pressure-balanced shields, Tunn. Undergr. Space Technol. 11 (1996) 165–173, https://doi.org/ 10.1016/0886-7798(96)00017-X. [33] A.A. Borovkov, Random walks and factorisation identities, in: A.A. Borovkov (Ed.), Probability Theory, Springer London, London, 2013. [34] C.J.C.H. Watkins, Learning from Delayed Rewards. Ph.D. Thesis, Oxford, 1989. [35] C.J.C.H. Watkins, P. Dayan, Q-learning, Mach. Learn. 8 (1992) 279–292, https:// doi.org/10.1007/BF00992698. [36] A.A. Javadi, M. Rezania, Applications of artificial intelligence and data mining techniques in soil modeling, Geomech. Eng. 1 (2009) 53–74, https://doi.org/ 10.12989/gae.2009.1.1.053. [37] M.A. Shahin, M.B. Jaksa, H.R. Maier, Recent advances and future challenges for artificial neural Systems in Geotechnical Engineering Applications, Adv. Artific. Neural Syst. 2009 (2009) 1–9, https://doi.org/10.1155/2009/308239. [38] H. Kinsley, Reinforcement Learning W/ Python. https://pythonprogramming.net /q-learning-reinforcement-learning-python-tutorial/, 2019. [39] R.H.R. Hahnloser, R. Sarpeshkar, M.A. Mahowald, R.J. Douglas, H.S. Seung, Digital selection and analogue amplification coexist in a cortex-inspired silicon circuit, Nature 405 (2000) 947–951, https://doi.org/10.1038/35016072. [40] X. Glorot, A. Bordes, Y. Bengio, Deep sparse rectifier neural networks, in: G. Gordon, D. Dunson, M. Dudik (Eds.), Proceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics, 2011, pp. 315–323. [41] Y. Lecun, L. Bottou, Y. Bengio, P. Haffner, Gradient-based learning applied to document recognition, Proc. IEEE 86 (1998) 2278–2324, https://doi.org/10.1109/ 5.726791. [42] D.H. HUBEL, T.N. WIESEL, Shape and arrangement of columns in cat’s striate cortex, J. Physiol. 165 (1963) 559–568, https://doi.org/10.1113/jphysiol.1963. sp007079. [43] F.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Wheel Loader Scooping Controller Using Deep Reinforcement Learning; DOI: 10.1109/access.2021.3056625; license: cc-by-sa.

[Assistant]
The ﬁrst involves writing Pulse Width Modulation (PWM) and direction commands to the actuators, whereas the other reads and publishes the current voltage of the linear actuator’s feedback potentiometer. 24148 VOLUME 9, 2021
[page 5] O. Azulay, A. Shapiro: Wheel Loader Scooping Controller Using Deep Reinforcement Learning V. SCOOPING CONTROLLER ARCHITECTURE In this section, we present our methodology for the DRL scooping motion controller. We describe the implementation of our complete learning system in a virtual environment and its training progression. A. PROBLEM FORMULATION We consider the combined bucket-scooping process of approaching and penetrating the pile, lifting, and then tilt- ing the bucket until breakout using UGV with a loading mechanism. Our goal is to ﬁnd a closed-loop policy using RL methods that maps sensor observations to robot action. We thus need a policy function at = f (st), where st is a parameterized vector of the current observed robot state using onboard sensor readings and at is the action composed of the lift, tilt, and throttle commands. Given the dynamics of the robot, we limit the linear velocity to 0.1m/s for moving towards and away from the pile, whereas lift and tilt actuators were not limited and could extract at any of their maximum velocities. B. SIMULATION ENVIRONMENT The training procedure of our model was implemented in a virtual 3D environment simulated by a Gazebo. We used a particle-based discrete representation of the pile. This simpli- ﬁcation of the real-world surfaces enabled the use of common physics simulators, which can maintain a close resemblance to reality. The number of particles impacts the real-time factor within the simulation considerably, which in turn represents the total run-time. In our environment, each pile of parti- cles contained 72 particles with radii of 3.4 cm and a mass of 0.3kg. Only spherical particles were used in this study. The simulation properties were carefully chosen to mimic the physics of the real robot as well as possible. The robot scooped gravel with a bulk density of 1.3 ton/m3 (bucket capacity is approximately 5[L], 7[kg]). Although the simu- lation was based on a simpliﬁed scenario which we found to be sufﬁcient to achieve our goal, other users could vary the environmental properties, such as the pile geometry, number of particles, and the maximum actuator load. In addition, users could choose to deﬁne the state according to other features (e.g., object orientation or angular velocities) as well as the authorized actions (e.g., discrete or continuous) for learning purposes. Also, users could build new models to independently collect data using the open-source code pro- vided here [17]. C. STATE REPRESENTATION The observation is the raw information provided by the robot sensors and the state is a compact depiction of this observa- tion that includes the information necessary for the robot to choose its actions. Following the robotic priors in [26], [27], we deﬁned the state using fused sensor measurements as a 12-dimensional vector consisting of the arm link positions (xt), the normalized mass of the excavated earth (wt), the FIGURE 5. Komodo robot with an arm mechanism and a pile of particles, simulated in the Gazebo physics engine. relative distance to the pile (dt), the state of the robot’s DoF (v, θT , θL) and their relative changes (1t). A visual depiction of these parameters is provided in Fig. 5. Deriving the amount of the excavated earth within the simulation environment was done directly using the simulation’s built-in functions. Measuring it in a real-world experiment could be done indi- rectly by approximation using the torque applied to the bucket joint as: τ = VSγ grcm (4) where VS is the excavated earth volume, γ is the material density, g is the gravitational acceleration and rcm is the approximate relative distance between the bucket CoM to the joint. The total force acting on the bucket can be decomposed into three main forces [6]: the shear force, the gravity force, and the remolding force. By deriving the amount of exca- vated earth from the applied torque, we included torque-to- mass extraction of the shearing force as the bucket penetrates the earth. Therefore, we limit our proposed controller to scooping earth with low adhesion, which is applicable to numerous earth-moving scenarios, since the main purpose of the wheel loader is the transportation of already excavated material. FIGURE 6.

Wheel Loader Scooping Controller Using Deep Reinforcement Learning


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant] Chollet (Ed.), Deep Learning with Python, Manning, Shelter Island, NY, 2018. [44] M. Abadi, A. Agarwal, P. Barham, E. Brevdo, Z. Chen, C. Citro, G.S. Corrado, A. Davis, J. Dean, M. Devin, S. Ghemawat, I. Goodfellow, A. Harp, G. Irving, M. Isard, Y. Jia, R. Jozefowicz, L. Kaiser, M. Kudlur, J. Levenberg, D. Man´e, R. Monga, S. Moore, D. Murray, C. Olah, M. Schuster, J. Shlens, B. Steiner, I. Sutskever, K. Talwar, P. Tucker, V. Vanhoucke, V. Vasudevan, F. Vi´egas, O. Vinyals, P. Warden, M. Wattenberg, M. Wicke, Y. Yu, X. Zheng, TensorFlow: Large-Scale Machine Learning on Heterogeneous Systems. https://www.tensor flow.org/, 2015. [45] F. Chollet, Others, Keras, 2015. [46] ¨Osterreichisches Normungsinstitut, Untertagebauarbeiten: Teil 1: Zyklischer Vortrieb, Wien 91.010.20; 93.020, 2019. [47] Z. Zhao, Y. Liang, X. Jin, Handling large-scale action space in deep Q network, in: 2018 International Conference on Artificial Intelligence and Big Data (ICAIBD), IEEE, Chengdu, 2018, pp. 93–96. [48] T. Zahavy, M. Haroush, N. Merlis, D.J. Mankowitz, S. Mannor, Learn what not to learn: action elimination with deep reinforcement learning, in: Advances in Neural Information Processing Systems 31 (NIPS 2018), Montreal, Canada, 2018, pp. 3566–3577. [49] ¨OGG, Guideline for the Geotechnical Design of Underground Structures with Conventional Excavation: Ground characterization and coherent procedure for the determination of excavation and support during design and construction. Translated from version 2.1, second.first, Salzburg, 2010. [50] H. Lauffer, Gebirgsklassifizierung für den Stollenbau, Geol. Bauwesen 24 (1958) 46–51. [51] P. Zhang, H. Li, Q.P. Ha, Z.-Y. Yin, R.-P. Chen, Reinforcement learning based optimizer for improvement of predicting tunneling-induced ground responses, Adv. Eng. Inform. 45 (2020) 101097, https://doi.org/10.1016/j.aei.2020.101097. [52] J. van Eldert, H. Schunnesson, D. Johansson, D. Saiang, Application of measurement while drilling technology to predict rock mass quality and rock support for Tunnelling, Rock Mech. Rock. Eng. 53 (2020) 1349–1358, https://doi. org/10.1007/s00603-019-01979-2. [53] A. Radinger, F. Fasching, G. Pack, I. Kreutzer, D. Kostial, Consistent exploration by probe drilling and TSWD through the example of the Koralm tunnel/Konsequente Vorauserkundung mittels Bohrungen und TSWD am Beispiel des Koralmtunnels, Geomechanik Tunnelbau 7 (2014) 540–550, https://doi.org/10.1002/ geot.201400038. G.H. Erharter et al.

Reinforcement learning based process optimization and strategy development in conventional tunneling


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Wheel Loader Scooping Controller Using Deep Reinforcement Learning; DOI: 10.1109/access.2021.3056625; license: cc-by-sa.

[Assistant]
FIGURE 12. Velocity, lift and tilt motion recorded in 5 consecutive scooping cycles. which thus illustrates the sensitivity of the resistive model to contacts. We also conducted another experiment to test the con- troller’s adaptability to different environments, but with typi- cal features such as different inclination angles of the terrain. A detailed comparison of the bucket tip trajectories (3 trials for each inclination angle) over a scooping cycle can be seen in Fig. 13 with three different pile inclination angles. Steeper pile inclination angles of the earth caused higher resistance force from the pile, leading to an earlier exit phase initialization, as can be seen in Fig 13. Overcoming different inclination angles supports the claim that our DRL-based controller can perform successfully in arbitrarily designed piles. The results of both experiments suggest that the tran- sition between different phases of the scooping process does FIGURE 13. Bucket tip trajectories over 3 different mean earth inclination angles (A). Applied torque at the bucket joint (B). FIGURE 14. Comparisons between the scooping process in the simulations and real-life experiments. TABLE 2. Comparison of scooping experiments. not depend on a single feature of the state, but rather on a combination of input features. The controller continued the ﬁlling phase until reaching the torque and depth thresholds, leading to an exit phase initialization, which appeared to be similar across all the attempts, as can be seen in Fig. 13. Comparing the DRL con- troller performance to the manually driven scooping attempts revealed differences between various processes of the scoop- ing cycle, as can be seen in Fig. 14. Due to the fact that we are not expert operators, it was extremely challenging to main- tain a successful smooth, fast continuous actions. The DRL controller achieved smoother action transitions and faster scooping cycles (average manual operation time was 10[sec], compared to 6 [sec] for the DRL-based controller). However, our manual attempts resulted in slightly better bucket weights, where the manual control resulted in an 24152 VOLUME 9, 2021
[page 9] O. Azulay, A. Shapiro: Wheel Loader Scooping Controller Using Deep Reinforcement Learning average of 6.1[kg] and the DRL controller resulted in an average of 5.5[kg] (9% difference), as shown in Table 2. Comparing the experimental results of our DRL controller to the simulation polices also pointed to differences between the phases of the scooping cycle. These included a difference between the penetration angle, that reached a maximum of 7.5 ± 1.0 [deg], higher loading cycles in the experiment, and a different trajectory within the second phase of the cycle, as illustrated in Fig. 14. These can be explained by the dis- crepancies between the simulation environment and the real- life experiments, such as particles vs. earth, action execution time, and mass approximation. VII. CONCLUSION In this article, a machine learning-based deep reinforce- ment learning scooping motion controller was applied for UGV with a custom-built scooping mechanism to perform scooping cycles with three DoF (lift, tilt, velocity) by fus- ing 12-dimensional sensor data as inputs to output actuator commands. The controller was trained using the actor-critic, off-policy Deep Deterministic Policy Gradient (DDPG) algo- rithm without any user-provided demonstrations. The learn- ing of the policy network is accomplished in a simpliﬁed simulation scenario, where the soil was represented as parti- cles. The scooping policy converged at the end of the training phase with an average loading cycle time of 5 seconds and an average ﬁll factor of 65% from full bucket capacity with an overﬁll. To investigate the performance and adaptability of our scooping controller, we tested our controller in several experiments with three different mean inclination angles of the earth. Overall, the DRL-based control exhibited good performance in terms of both achieved visual bucket ﬁll with varying scooped earth weights of 4.1 −7.2[kg], and 5.1 −7.1[sec] cycle time. Comparisons between the man- ual and our DRL controller indicated that human control led to the highest average bucket load. Although the DRL controller may not yet be on a par with human control for the highest volume, the DRL controller here did achieve smoother trajectories and lower cycle times than the man- ual control.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant]
We give detailed lists of the possible actions in Section 3.2.1 and explain the exact content of the state and the rewards in the Sections 3.3.1 and 3.3.2 respectively. Fig. 1. Schematic diagram of cyclical and sequential processes in conventional tunnel excavation. The size of loops represents a qualitative hierarchy with smaller loops being components of bigger loops. The excavation sequence decision – loop is given in bold, as this will be the main focus of this paper. G.H. Erharter et al.
[page 3] Automation in Construction 127 (2021) 103701 3 2.1. Simplifications As shown in Fig. 1, conventional tunneling is a complex process that consists of several subprocesses which again have subprocesses etc. Attempting to formulate this whole system as a RL problem with all involved details is out of the scope of an initial study which should serve as the basis for future research. Additionally, it must also be considered that RL is still at the very beginning of practical applications. Conse quently, we had to take several simplifications of conventional tunneling to reduce the general complexity of the simulation and the size of the state - and action space. Still, we consider modelling the major decision- loop and the actions and rewards, not to be too far from a realistic tunnel approach, and will contribute with important insights to the use of RL in the optimization of the conventional tunnel cycle. Some important simplifications in comparison to reality were made in this study: ▪ Available partial excavation methods are restricted to either top heading - or a combined bench and invert excavation (described as “bench excavation” hereafter) with a specific tunnel geometry. ▪ Already excavated parts are considered to be stable and the process of tunnel lining installation is not dealt with in the simulation. ▪ If installed, support ahead of the face always consists of 10 m long face anchors. The supported area ahead is stable and excavation within that stable area cannot lead to failure. No other types of rock support are considered, such as radial bolts and shotcrete. ▪ We evaluate if stable conditions are at hand in the excavation area by the face pressure equation for open face tunneling after [29] (see Eq. (3) in Section 3.3.2). We chose this analytical solution as it is a computationally efficient way for a stability assessment that tells if stability is given or not at a certain ground type. Although this is only one aspect of the tunnel stability considerations, we see this approach as sufficient for the present initial study. ▪ The stability assessment considers the cross-sectional area only and no longitudinal effects. ▪Rockmass quality is reduced to few mechanical values repre senting “favorable” or “unfavorable” rockmass conditions. ▪ There are only two available advance lengths (i.e. the length of one blasting round/round of excavation) with 2 and 4 m each. ▪ Information from probe drilling or deformation monitoring is not simulated and used in the decision process. That means that the agent has no information about the rockmass ahead of the face or no information about eventual deformation behind the face. 3. A simulation of conventional tunneling As the main goal of this study is to train a RL agent to execute an excavation sequence as efficiently as possible, we designed a simulation of such a scenario. The simulation consists of a longitudinal tunnel section of a specific length (tl), where two different types of ground conditions can occur – one favorable and one unfavorable. Before the excavation, the agent is unaware of the distribution of ground types and the distribution of the ground types is only revealed by the excavation itself. The ultimate goal of the agent is to achieve a breakthrough of both the top heading and the bench of the tunnel or in other words, the po sition of the top heading excavation (posth) and the position of the bench excavation (posbi) must be greater than, or equal to tl. During the exca vation the agent can choose from different actions, e.g. top heading excavation with 2 m advance length, bench excavation with 4 m advance length and installation of face support etc. 3.1. Geotechnical scenario For this study’s simulation we have chosen the following tunneling scenario: The total length of the tunnel (tl) is 200 m, as this does not lead to an excessively large state space in the RL model (see Section 3.3.1) but is still a realistic length.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Cutterhead and Cutting Tools Configurations in Coarse Grain Soils; DOI: 10.2174/1874836801711010182; license: cc-by.

[Assistant]
The single disc cutters have a single point of contact at the face and tend to skid or be stagnant in the soft or not compact soils, wearing flat on one side and making excavation of the over-sized particles difficult. Frequent work of changing cutting tools at more time and expenses has to be executed during the whole tunnel construction. Whereas, disc cutters with tungsten carbide inserts can excavate cobbles and boulders with a higher efficiency because they have multiple points of contact at the face. The multiple buttons act like traction, allowing the carbide cutters to ‘grip’ in soft or loosened ground and keep rotating. As compared to performances of the twin disc cutters, it was found that the twin disc cutters were more easily worn out than single disc cutters when building a railway tunnel in Beijing sandy gravel/cobble soils with boulders [32]. So, it is recommended that the plate type cutterheads equipped with the chip inserted single disc cutters in combination with the more aggressive soft ground tools are adopted when boring tunnels in the water-rich coarse grain soils. Fig. (20). Clogging at the cutterhead center. 4.4. Reasonable Movement of Shield Machines As cutting tools move in the excavation face they are taking up highly viable loads. The wear and failure of the tools is the result of such cyclic loadings. As revealed in Figs. (15, 17), dropping off of tips and fractures of cutting tools are prevailing damages caused by cobbles or boulders when they abruptly hit these over-sized particles. To mitigate the adverse effects, deciding the slow and reasonable movements (advance, rotation) of a shield machine is of great importance. Particularly for hard boulder-laden ground when using disc cutters the penetration rate per cutterhead rotation should be well regulated and restricted to avoid that boulders will be ripped-out of the soil matrix, endangering the stability of the cutting face and damaging cutterhead and cutting tools. For a spoke type cutterhead shield machine, the well planned sensible and slow movements for the specific ground conditions can greatly reduce the damages to precut bits as well as to scrapers caused by collisions with big cobbles or boulders. 4.5. Robust Shield Machines Again Wears The wears and damages to cutting tools, cutterhead and other components do exist if shield tunneling will be performed in the coarse grain soils. Particularly, when a long tunnel will be constructed, it is absolutely necessary to take whatever, state-of-the-art measures to reduce the adverse influences. In recent years, large improvements have been achieved in relation to the cutting tool durability and wear protection on cutterhead as well as other components of a shield machine [33]. The highlights are more specialized, robust wear resistant tools. These robust tools, such as pressure compensated disc cutters of monobloc design including options with carbide inserts, soft ground tools (scrapers, rippers) using carbide cutting edges or inserts in combinations with hardfacing, will allow for a longer lifespan of the cutting tools under the coarse grain ground conditions. Using lately developed twin layer plating materials, overall replaceable wear protections of the cutterhead in weld-on or bolted-on designs can also be effectively applied to the shield machines in coarse grain soils [34, 35]. The designs consist of wedges to protect the cutters, wear
[page 17] Cutterhead and Cutting Tools Configurations The Open Construction and Building Technology Journal, 2017, Volume 11 197 plates for the face, wear protection at the cutterhead periphery, grill bars to protect the cutterhead periphery, and grill bars to protect the scrapers from big blocks and control the maximum block size to enter the cutterhead. 4.6. Interventions for Changing Tools Interventions and chamber access for repairing tools and other components are sometimes indispensable to some demanding and challenging works of the tunnel construction in coarse grain soils. Compared with changing cutting tools under compressed air condition, changing tools in free air, or “atmospheric cutter change” based on a hollow cutterhead structure for a large diameter shield machine [33, 35], is a better choice.

Cutterhead and Cutting Tools Configurations in Coarse Grain Soils


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Detection of Ionospheric Scintillation Based on XGBoost Model Improved by SMOTE-ENN Technique; DOI: 10.3390/rs13132577; license: cc-by-sa.

[Assistant] [page 1] remote sensing Article Detection of Ionospheric Scintillation Based on XGBoost Model Improved by SMOTE-ENN Technique Mengying Lin 1 , Xuefen Zhu 1,*, Teng Hua 1, Xinhua Tang 1, Gangyi Tu 2 and Xiyuan Chen 1   Citation: Lin, M.; Zhu, X.; Hua, T.; Tang, X.; Tu, G.; Chen, X. Detection of Ionospheric Scintillation Based on XGBoost Model Improved by SMOTE-ENN Technique. Remote Sens. 2021, 13, 2577. https://doi.org/ 10.3390/rs13132577 Academic Editor: Michael E. Gorbunov Received: 1 June 2021 Accepted: 29 June 2021 Published: 1 July 2021 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional afﬁl- iations. Copyright: © 2021 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). 1 Key Laboratory of Micro-Inertial Instrument and Advanced Navigation Technology of Ministry of Education, Sipailou Campus, School of Instrument Science and Engineering, Southeast University, Nanjing 210096, China; linmengying@seu.edu.cn (M.L.); 220203499@seu.edn.cn (T.H.); xinhua.tang@seu.edu.cn (X.T.); chxiyuan@seu.edu.cn (X.C.) 2 School of Electronic and Information Engineering, Nanjing University of Information Science and Technology, Ning Liu Road, Nanjing 210044, China; tugangyi@nuist.edu.cn * Correspondence: zhuxuefen@seu.edu.cn; Tel.: +86-136-4516-1372 Abstract: Ionospheric scintillation frequently occurs in equatorial, auroral and polar regions, posing a threat to the performance of the global navigation satellite system (GNSS). Thus, the detection of ionospheric scintillation is of great signiﬁcance in regard to improving GNSS performance, es- pecially when severe ionospheric scintillation occurs. Normal algorithms exhibit insensitivity in strong scintillation detection in that the natural phenomenon of strong scintillation appears only occasionally, and such samples account for a small proportion of the data in datasets relative to those for weak/moderate scintillation events. Aiming at improving the detection accuracy, we proposed a strategy combining an improved eXtreme Gradient Boosting (XGBoost) algorithm by using the synthetic minority, oversampling technique and edited nearest neighbor (SMOTE-ENN) resampling technique for detecting events imbalanced with respect to weak, medium and strong ionospheric scintillation. It outperformed the decision tree and random forest by 12% when using imbalanced training and validation data, for tree depths ranging from 1 to 30. For different degrees of imbalance in the training datasets, the testing accuracy of the improved XGBoost was about 4% to 5% higher than that of the decision tree and random forest. Meanwhile, the testing results for the improved method showed signiﬁcant increases in evaluation indicators, while the recall value for strong scintillation events was relatively stable, above 90%, and the corresponding F1 scores were over 92%. When testing on datasets with different degrees of imbalance, there was a distinct increase of about 10% to 20% in the recall value and 6% to 11% in the F1 score for strong scintillation events, with the testing accuracy ranging from 90.42% to 96.04%. Keywords: GNSS; ionospheric scintillation detection; XGBoost; SMOTE-ENN 1. Introduction The ionosphere, the atmosphere at about 60 to 1000 km from the ground, is modulated by the ionizing effects of solar radiation, particle precipitation and the geomagnetic ﬁeld. There are some typical ionospheric phenomena, such as the equatorial ionization anomaly (EIA) and equatorial plasma bubbles at low attitudes [1–3], as well as the tongue of ion- ization at high latitudes [4,5], by which radio waves such as global navigation satellite system (GNSS) signals may be severely affected. When passing through the ionospheric irregularities, the signals are plagued with rapid ﬂuctuation, phase shifts, delay, multi- path, and even loss of tracking loop. More seriously, this phenomenon of ionospheric scintillation occurs more frequently and severely in both low-latitude regions and polar regions, compromising positioning accuracy and continuity [6,7]. In high-latitude regions, the occurrence of ionospheric scintillation is more apparent during geomagnetic storms, and the formation of irregular structures and ionospheric scintillation activities appears to Remote Sens.

Detection of Ionospheric Scintillation Based on XGBoost Model Improved by SMOTE-ENN Technique


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant]
We give detailed lists of the possible actions in Section 3.2.1 and explain the exact content of the state and the rewards in the Sections 3.3.1 and 3.3.2 respectively. Fig. 1. Schematic diagram of cyclical and sequential processes in conventional tunnel excavation. The size of loops represents a qualitative hierarchy with smaller loops being components of bigger loops. The excavation sequence decision – loop is given in bold, as this will be the main focus of this paper. G.H. Erharter et al.
[page 3] Automation in Construction 127 (2021) 103701 3 2.1. Simplifications As shown in Fig. 1, conventional tunneling is a complex process that consists of several subprocesses which again have subprocesses etc. Attempting to formulate this whole system as a RL problem with all involved details is out of the scope of an initial study which should serve as the basis for future research. Additionally, it must also be considered that RL is still at the very beginning of practical applications. Conse quently, we had to take several simplifications of conventional tunneling to reduce the general complexity of the simulation and the size of the state - and action space. Still, we consider modelling the major decision- loop and the actions and rewards, not to be too far from a realistic tunnel approach, and will contribute with important insights to the use of RL in the optimization of the conventional tunnel cycle. Some important simplifications in comparison to reality were made in this study: ▪ Available partial excavation methods are restricted to either top heading - or a combined bench and invert excavation (described as “bench excavation” hereafter) with a specific tunnel geometry. ▪ Already excavated parts are considered to be stable and the process of tunnel lining installation is not dealt with in the simulation. ▪ If installed, support ahead of the face always consists of 10 m long face anchors. The supported area ahead is stable and excavation within that stable area cannot lead to failure. No other types of rock support are considered, such as radial bolts and shotcrete. ▪ We evaluate if stable conditions are at hand in the excavation area by the face pressure equation for open face tunneling after [29] (see Eq. (3) in Section 3.3.2). We chose this analytical solution as it is a computationally efficient way for a stability assessment that tells if stability is given or not at a certain ground type. Although this is only one aspect of the tunnel stability considerations, we see this approach as sufficient for the present initial study. ▪ The stability assessment considers the cross-sectional area only and no longitudinal effects. ▪Rockmass quality is reduced to few mechanical values repre senting “favorable” or “unfavorable” rockmass conditions. ▪ There are only two available advance lengths (i.e. the length of one blasting round/round of excavation) with 2 and 4 m each. ▪ Information from probe drilling or deformation monitoring is not simulated and used in the decision process. That means that the agent has no information about the rockmass ahead of the face or no information about eventual deformation behind the face. 3. A simulation of conventional tunneling As the main goal of this study is to train a RL agent to execute an excavation sequence as efficiently as possible, we designed a simulation of such a scenario. The simulation consists of a longitudinal tunnel section of a specific length (tl), where two different types of ground conditions can occur – one favorable and one unfavorable. Before the excavation, the agent is unaware of the distribution of ground types and the distribution of the ground types is only revealed by the excavation itself. The ultimate goal of the agent is to achieve a breakthrough of both the top heading and the bench of the tunnel or in other words, the po sition of the top heading excavation (posth) and the position of the bench excavation (posbi) must be greater than, or equal to tl. During the exca vation the agent can choose from different actions, e.g. top heading excavation with 2 m advance length, bench excavation with 4 m advance length and installation of face support etc. 3.1. Geotechnical scenario For this study’s simulation we have chosen the following tunneling scenario: The total length of the tunnel (tl) is 200 m, as this does not lead to an excessively large state space in the RL model (see Section 3.3.1) but is still a realistic length.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant] [page 1] Automation in Construction 127 (2021) 103701 Available online 17 April 2021 0926-5805/© 2021 The Author(s). Published by Elsevier B.V. This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/). Reinforcement learning based process optimization and strategy development in conventional tunneling Georg H. Erharter a,*, Tom F. Hansen b, Zhongqiang Liu b, Thomas Marcher a a Graz University of Technology, Institute for Rock Mechanics and Tunnelling, Rechbauerstraße 12, Graz, Austria b Norwegian Geotechnical Institute, Oslo, Norway A R T I C L E I N F O Keywords: Conventional tunneling Reinforcement learning Tunnel excavation strategy Machine learning Excavation sequences A B S T R A C T Reinforcement learning (RL) - a branch of machine learning - refers to the process of an agent learning to achieve a certain goal by interaction with its environment. The process of conventional tunneling shows many similar ities, where a geotechnician (agent) tries to achieve a breakthrough (goal) by excavating the rockmass (envi ronment) in an optimum way. In this paper we present a novel RL based framework for strategy development for conventional tunneling. We developed a virtual environment with the goal of a tunnel breakthrough and with a deep Q-network as the agent’s architecture. It can choose from different excavation sequences to reach that goal and learns to do so in an economical and safe way by getting feedback from a specially designed reward system. Result analyses show that the optimal policies have great similarities to current practices of sequential tunneling and the framework has the potential to discover new tunneling strategies. 1. Introduction Digitalization in tunneling is an ongoing process including topics like Artificial Intelligence (AI) technology [1], Building Information Modelling [2] or embedded strain measurements in shotcrete lining [3]. Machine Learning (ML) applications in particular have so far mostly focused on classification tasks based on supervised ML: e.g. [4–6] apply artificial neural networks (ANN) for rockmass behavior classification of tunnel boring machine (TBM) data; [7] use supervised learning methods for automatic work progress identification in NATM (New Austrian Tunneling Method) tunneling; see [8–11] for state-of-the-art reviews on this topic. Applications of unsupervised ML are fewer in number and often related to the search for less dependency on labelled datasets or more objectiveness (e.g. [12,13]). By the time of this writing reinforcement learning (RL) is mostly a matter of research, but already shows mature problem-solvers for game like scenarios [14,15]. Currently, there is a transition from academia to real-world prototypes, with RL-examples like the optimization of a manufacturing process [16], for real-time steering of hydrocarbon drilling [17,18], in optimization of power grids [19] and control systems in general, for AUVs [20], and in robotics and other autonomous vehi cles. To our knowledge, however, there is no published application of RL to geotechnics in general or tunneling in particular. Today’s conventional tunneling – sometimes referred to as “drill and blast tunneling” or “sequential excavation method” – is the classical way of tunnel construction and is the product of more than a century of engineering experience [21]. Great flexibility to adapt to changing ground conditions is one of the main benefits of this type of tunneling, but technical and economic success of the excavation is dependent on the experience of the involved engineers and workers. Albeit experience is undoubtedly valuable, depending on it sometimes goes along with simple repetition of “proven ways” or even negligence of innovation. Furthermore, developments in conventional tunneling often have a strong connection to their nation of origin (e.g. Austria: New Austrian Tunnelling Method [22], Norway: Norwegian Tunneling Method [23], Italy: New Italian Tunneling Method [24,25], etc.) which raises concerns about biased researchers and engineers. The goal of this study is to take a first step in the direction of a conceptual RL-model for optimum decision making in conventional tunneling that is as free as possible of conser vatism and national biases. Furthermore, as shown in the development of the RL-agent AlphaGO [15], RL systems have the potential to find new solutions to old problems and thus discover unimagined strategies.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Two-phase multi-model automatic brain tumour diagnosis system from magnetic resonance images using convolutional neural networks; DOI: 10.1186/s13640-018-0332-4; license: cc-by-sa.

[Assistant]
2 The architecture of the CNN used in this study. The name of each network layer is provided. The figure shows a snapshot of the whole network architecture To facilitate the convolution process, the dimensions of the convolutional kernel and the CNN input should be matched. If the CNN is a multidimensional array, then the kernel should be a multidimensional array as well. If the CNN input is a two-dimensional image [38], a two-dimensional kernel w is used, as presented in Eq. 1. The output of the convolutional layers, i.e., layers 2 and 6, will apply a sigmoidal activation function to the obtained convolutional output y in Eq. 1 to provide the new output f modelled as function of the input y with Eqs. 2 or 3: f (y) = tanh(y) (2) f (y) = 1 1 + e−y (3) The output of the ReLU layers, i.e., layers 3 and 7, will apply a linear activation function to the neuron output y as shown in Eq. 4: f (y) = max(0, y) (4) Layers 4 and 8 are the cross-channel normalization lay- ers with five channels. The 3 × 3 max-pooling layers are layers 5 and 9. These layers are very simple because they take a K ×K region and output a single value, and then the output is an N K × N K layer. Layer 9 provides the extracted features in a dimension of 1 × 43264, which are then fed to the SVM to carry out the image classification. 3.1.3 Error-correcting output codes support vector machines (ECOC-SVMs) Dietterich and Bakiri presented an error-correcting out- put codes (ECOC) approach to correct the data error when input enters into a channel. The approach uses error-correcting codes to convert the binary classifiers to a multi-class classification [39]. It has been successfully used in many applications, such as face recognition [40], face verification [41], text recognition [42], and digital manuscript classification [43]. Error-correcting output codes support vector machines (ECOC-SVMs) is a method that combines the SVMs and ECOC. It was proposed to solve the online identification and feature extraction problems. When applying ECOC- SVMs to multi-classification, the k classes classification is transformed to the L classes classification by the ECOC matrix. ECOC-SVM functions according to the following steps [44]: 1 Training: The classes are represented in two subsets (0 or 1) for each row of the coding matrix using the SVM binary partition. The SVM takes the row with a value of 0 as the first class and the row with a value of 1 as the second class. The corresponding class assigns a codeword for each column of the matrix. All of the L classes are trained according to rows of the ECOC matrix, where L is the dimensionality of the new feature space and the length of the codeword. 2 Testing: The test data is fed into the learned classifier. The L classifiers recognize the samples and generate output vector Z. The hamming distance
[page 6] Abd-Ellah et al. EURASIP Journal on Image and Video Processing (2018) 2018:97 Page 6 of 10 Fig. 3 Representative images from the MRIs database that was used in the tumour detection phase. Normal MRIs without brain tumours are in the top row, while abnormal images with brain tumours present are in the bottom row between each vector in the output vector matrix and code matrix is shown in Eq. 5. The classifier chooses the class depending on the minimal distance. ci = argmind(Z, Hi) = L  j=1 |Zj −Hi,j|, i = 1, 2, ..., k (5) Table 3 The obtained number of features, the consumed feature extraction time, and the accuracy of the different CNN layers Layer number Evaluation metrics No. of features Feature extraction time Accuracy (%) Layer 2 (conv) 290400 0.24 69.16 Layer 3 (ReLU) 290400 0.25 84.58 Layer 4 (norm) 290400 0.33 68.28 Layer 5 (pool) 69984 0.35 66.96 Layer 6 (conv) 186624 0.78 69.16 Layer 7 (ReLU) 186624 0.80 48.01 Layer 8 (norm) 186624 0.83 77.97 Layer 9 (pool) 43264 0.85 99.55 Layer 10 (conv) 64896 1.20 49.33 Layer 11 (ReLU) 64896 1.21 81.93 Layer 12 (conv) 64896 1.40 65.63 Layer 13 (ReLU) 64896 1.41 98.23 Layer 14 (conv) 43264 1.59 88.54 Layer 15 (ReLU) 43264 1.60 96.47 The table indicates that the optimum number of layers with respect to the consumed time and the achieved accuracy is 9 where H is the code matrix with i rows and j columns. 3.2 Brain tumour localization phase Since the BraTS 2013 database includes 3D MRIs, we used its MRIs slice by slice as inputs for the CNN. Our method handles each slice as a 2D image [29], and T2 images were selected only with segmented images.

Two-phase multi-model automatic brain tumour diagnosis system from magnetic resonance images using convolutional neural networks


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant]
The tunnel’s cross section has a total area of 91.31m2 and a height of 10 m with 58.56m2 and 32.75m2 being the areas of the top heading - and the bench and invert excavation respec tively (see Fig. 3). From these areas, equivalent diameters (D, i.e. the diameter of a circle with that area) of 8.41 m and 6.46 m can be computed for both parts of the excavation. There is a penalty if the distance between the top heading’s- and the bench’s tunnel face is too big. This distance (distmax) is set to be 50 m and the idea behind this is, that in many real projects – especially in soil conditions and long tunnels -, the top heading cannot be driven indefi nitely long ahead of the bench, because of safety reasons, necessities to Fig. 2. Simplified schematic plot of components of drill and blast tunneling (grey) as a reinforcement learning process - TunnRL. Fig. 3. Tunnel cross section of the given simulation. G.H. Erharter et al.
[page 4] Automation in Construction 127 (2021) 103701 4 have a fast final ring closure, excavate cross cuts or general construction logistics. We defined two ground types (gt) for the simulation and the relevant ground properties are given in Table 1. Both are considered to be of homogeneous, isotropic and continuous nature and one can imagine them as a type of hard soil/soft rock (HSSR) material [30,31]. With the chosen parameters, gt1 represents “unfavorable” ground conditions and gt2 represents “favorable” ground conditions. The condition for a gt to be favorable/unfavorable is based on the ground properties in combi nation with the given tunnel geometry (see above) as evaluated by the chosen stability assessment criterion (see Section 3.3.2). The simulated tunnel is situated above the groundwater table and the permeability is set to 10−5 m/s for both gt. Choosing the given properties to define the gt, is closely connected to the way the stability assessment of the excavation is done (see Eq. (3) in Section 3.3.2). The permeability of 10−5 justifies the use of Eq. (3) for a stability assessment as according to [32] conditions are considered to be drained when the permeability is above 10−7 to 10−6 m/s. Nevertheless, if ground types with lower permeability/undrained conditions are to be used for the simulation, appropriate solutions for stability assessments must be chosen. Implementing more sophisticated stability assessments that take phenomena like ground water conditions into account is desirable but must be done with care as this heavily influences the overall performance of the tunneling simulation and the RL itself. The outlook and discussion of stability assessments in Section 7 goes into more detail on this topic. Using random walks with barriers [33], we created unique, 210 m long geological sections with a decimeter resolution (see Section 3.3.1 for why the sections are 210 and not 200 m long). We created a ground type-vector (gt-vector) of 2100 datapoints by scaling the random walk between 0 and 1, rounding to full numbers and using 0 as gt1 and 1 as gt2 (see Fig. 4 bottom row). To transform the gt-vector to a full geological section with one row for the top heading and one for the bench excavation, the vector is horizontally duplicated to an array of 2 × 2100 datapoints. + 1 is then added to the array so that the number 1 represents gt1, number 2 gt2 and number 0 represents the unexcavated part of the tunnel. The top row of Fig. 4 shows a visualization of such an array, where posth and posbi are at 165 and 125 m respectively. Given the complexity of simulating the process of conventional tunneling (see Section 2), we chose not to complicate the scenario by introducing more gt. However, increasing the number of gt can easily be done with the above described random walk based approach. For example, if it was necessary to simulate four gt then the values of the random walk must be split into four within the boundaries: gt1 < 0.25, 0.25 ≤gt2 < 0.5, 0.5 ≤gt3 < 0.75 and 0.75 ≥gt4. 3.2. Agent Translating the above described geotechnical scenario to RL, the geotechnician who observes the state of the construction and rockmass behavior and makes decisions based on this information, now becomes the “RL agent”. Due to the state-complexity of the problem, we chose a deep Q-network (DQN) as the RL agent.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Intelligent Classification of Surrounding Rock of Tunnel Based on 10 Machine Learning Algorithms; DOI: 10.3390/app12052656; license: cc-by-sa.

[Assistant] [page 1]   Citation: Zhao, S.; Wang, M.; Yi, W.; Yang, D.; Tong, J. Intelligent Classiﬁcation of Surrounding Rock of Tunnel Based on 10 Machine Learning Algorithms. Appl. Sci. 2022, 12, 2656. https://doi.org/10.3390/ app12052656 Academic Editors: Mingfeng Lei, Chenjie Gong and Xianda Shen Received: 7 February 2022 Accepted: 27 February 2022 Published: 4 March 2022 Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional afﬁl- iations. Copyright: © 2022 by the authors. Licensee MDPI, Basel, Switzerland. This article is an open access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https:// creativecommons.org/licenses/by/ 4.0/). applied sciences Article Intelligent Classiﬁcation of Surrounding Rock of Tunnel Based on 10 Machine Learning Algorithms Siguang Zhao 1,2, Mingnian Wang 1,2, Wenhao Yi 1,2, Di Yang 1,2 and Jianjun Tong 1,2,* 1 Key Laboratory of Transportation Tunnel Engineering of Ministry of Education, Southwest Jiaotong University, Chengdu 610031, China; 2019310044@my.swjtu.edu.cn (S.Z.); wangmingnian@swjtu.edu.cn (M.W.); 1256635900@my.swjtu.edu.cn (W.Y.); 2020210212@my.swjtu.edu.cn (D.Y.) 2 School of Civil Engineering, Southwest Jiaotong University, Chengdu 610031, China * Correspondence: jjtong@swjtu.edu.cn; Tel.: +86-135-5031-1331 Abstract: The quality evaluation of the surrounding rock is the cornerstone of tunnel design and construction. Previous studies have conﬁrmed the existence of a relationship between drilling parameters and the quality of surrounding rock. The application of drilling parameters to the intelligent classiﬁcation of surrounding rock has the natural advantages of automatic information collection, real-time analysis, and no extra work. In this work, we attempt to establish the intelligent surrounding rock classiﬁcation model and software system driven by drilling parameters. We collected 912 samples containing four drilling parameters (penetration velocity, hammer pressure, rotation pressure, and feed pressure) and three surrounding rock (grade-III, grade-IV, and grade-V). Based on the python machine learning toolkit (Scikit-learn), 10 types of supervised machine learning algorithms were used to train the intelligent surrounding rock classiﬁcation model with the model parameter selection technology of grid search cross validation. The results show that the average accuracy is 0.82, which proves the feasibility of this method. Finally, the tunnel surrounding rock intelligent classiﬁcation system was established based on three models with better comprehensive performance among them. The classiﬁcation accuracy of the system was 0.87 in the tunnel test section, which indicates that the system has good generalization performance and practical value. Keywords: drill and blast tunnel; machine learning; measure-while-drilling; drilling parameters; intelligent surrounding rock classiﬁcation model 1. Introduction Unlike aboveground engineering, in tunnel engineering, tunnels are built under- ground. Tunnel design and construction are closely related to the stratum where it is buried. The quality evaluation of the surrounding rock is the cornerstone of tunnel design [1–3] and construction. Surrounding rock classiﬁcation is a main evaluation method for the surrounding rock quality. It usually collects the information of surrounding rock by one or more means, and ﬁnally gives a comprehensive evaluation index based on speciﬁc rule. It can reﬂect the strength characteristics and deformation characteristics of the surrounding rock and stability characteristics of the tunnel face, and can be directly used to guide the tunnel design and construction. The classiﬁcation of surrounding rock is a common method for the surrounding rock quality evaluation of tunnels in various countries. This method was started in Europe in 1774. After more than two hundred years of development, there have been hundreds of methods employed for this purpose, such as the Q-value method [4], the rock mass rating (RMR) method [5], and the surrounding rock basic quality index (BQ) method [6]. At present, the methods used to classify the rock surrounding tunnels are mainly qualitative, with quantitative veriﬁcation using laboratory tests requiring extensive profes- sional knowledge and engineering experience. Therefore, there is some randomness in the Appl. Sci.

Intelligent Classification of Surrounding Rock of Tunnel Based on 10 Machine Learning Algorithms


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: How to evaluate performance of prediction methods? Measures and their interpretation in variation effect analysis; DOI: 10.1186/1471-2164-13-s4-s2; license: cc-by.

[Assistant]
Vihinen BMC Genomics 2012, 13(Suppl 4):S2 http://www.biomedcentral.com/1471-2164/13/S4/S2 Page 9 of 10
[page 10] This article has been published as part of BMC Genomics Volume 13 Supplement 4, 2012: SNP-SIG 2011: Identification and annotation of SNPs in the context of structure, function and disease. The full contents of the supplement are available online at http://www.biomedcentral.com/ bmcgenomics/supplements/13/S4. Author details 1Institute of Biomedical Technology, FI-33014 University of Tampere, Finland. 2BioMediTech, Tampere, Finland. 3Department of Experimental Medical Science, Lund University, SE-221 84 Lund, Sweden. Competing interests The author declares that they have no competing interests in relation to the SNP-SIG issue article. Published: 18 June 2012 References 1. Thusberg J, Vihinen M: Pathogenic or not? And if so, then how? Studying the effects of missense mutations using bioinformatics methods. Hum Mutat 2009, 30:703-714. 2. Khan S, Vihinen M: Performance of protein stability predictors. Hum Mutat 2010, 31:675-684. 3. Potapov V, Cohen M, Schreiber G: Assessing computational methods for predicting protein stability upon mutation: good on average but not in the details. Protein Eng Des Sel 2009, 22:553-560. 4. Desmet F, Hamroun G, Collod-Beroud G, Claustres M, Beroud C: Res. Adv. in Nucleic Acid Research. Global Research Network; 2010. 5. Thusberg J, Olatubosun A, Vihinen M: Performance of mutation pathogenicity prediction methods on missense variants. Hum Mutat 2011, 32:358-368. 6. Moult J, Fidelis K, Kryshtafovych A, Tramontano A: Critical assessment of methods of protein structure prediction (CASP)–round IX. Proteins 2011, 79(Suppl 10):1-5. 7. Rodrigues AP, Grant BJ, Godzik A, Friedberg I: The 2006 automated function prediction meeting. BMC Bioinformatics 2007, 8(Suppl 4):S1-4. 8. Wodak SJ: From the Mediterranean coast to the shores of Lake Ontario: CAPRI’s premiere on the American continent. Proteins 2007, 69:697-698. 9. McClure MA, Vasi TK, Fitch WM: Comparative analysis of multiple protein- sequence alignment methods. Mol Biol Evol 1994, 11:571-592. 10. Thompson JD, Plewniak F, Poch O: BAliBASE: a benchmark alignment database for the evaluation of multiple alignment programs. Bioinformatics 1999, 15:87-88. 11. Mizuguchi K, Deane CM, Blundell TL, Overington JP: HOMSTRAD: a database of protein structure alignments for homologous families. Protein Sci 1998, 7:2469-2471. 12. Raghava GP, Searle SM, Audley PC, Barber JD, Barton GJ: OXBench: a benchmark for evaluation of protein multiple sequence alignment accuracy. BMC Bioinformatics 2003, 4:47. 13. Edgar RC: MUSCLE: multiple sequence alignment with high accuracy and high throughput. Nucleic Acids Res 2004, 32:1792-1797. 14. Van Walle I, Lasters I, Wyns L: SABmark–a benchmark for sequence alignment that covers the entire known fold space. Bioinformatics 2005, 21:1267-1268. 15. Orengo CA, Michie AD, Jones S, Jones DT, Swindells MB, Thornton JM: CATH–a hierarchic classification of protein domain structures. Structure 1997, 5:1093-1108. 16. Kolodny R, Koehl P, Levitt M: Comprehensive evaluation of protein structure alignment methods: scoring by geometric measures. J Mol Biol 2005, 346:1173-1188. 17. Lo Conte L, Ailey B, Hubbard TJ, Brenner SE, Murzin AG, Chothia C: SCOP: a structural classification of proteins database. Nucleic Acids Res 2000, 28:257-259. 18. Sonego P, Pacurar M, Dhir S, Kertesz-Farkas A, Kocsor A, Gaspari Z, Leunissen JA, Pongor S: A protein classification benchmark collection for machine learning. Nucleic Acids Res 2007, 35:D232-236. 19. Hwang H, Vreven T, Janin J, Weng Z: Protein-protein docking benchmark version 4.0. Proteins 2010, 78:3111-3114. 20. Cope LM, Irizarry RA, Jaffee HA, Wu Z, Speed TP: A benchmark for Affymetrix GeneChip expression measures. Bioinformatics 2004, 20:323-331. 21. Zhu Q, Miecznikowski JC, Halfon MS: Preferred analysis methods for Affymetrix GeneChips. II. An expanded, balanced, wholly-defined spike- in dataset. BMC Bioinformatics 2010, 11:285. 22. Gray J: The Benchmark Handbook for Database and Transaction Systems. Morgan Kaufmann; 1993. 23. Aniba MR, Poch O, Thompson JD: Issues in bioinformatics benchmarking: the case study of multiple sequence alignment. Nucleic Acids Res 2010, 38:7353-7363. 24. Berman HM, Westbrook J, Feng Z, Gilliland G, Bhat TN, Weissig H, Shindyalov IN, Bourne PE: The Protein Data Bank. Nucleic Acids Res 2000, 28:235-242. 25.

How to evaluate performance of prediction methods? Measures and their interpretation in variation effect analysis


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Spectral Angle Mapping and AI Methods Applied in Automatic Identification of Placer Deposit Magnetite Using Multispectral Camera Mounted on UAV; DOI: 10.3390/min12020268; license: cc-by-sa.

[Assistant]
SAM Analysis Outputs From the three UAV ﬂight elevations, multispectral images were analyzed and the results show that by varying the reference cosine similarity threshold values, a precise mapping of the area can be attained. By altering their threshold values, Shafri et al. [17] too were able to attain better subject separation. It is clear to see that the SAM analysis performed well across all the three UAV ﬂight elevations from visual analysis of Figure 5. When the threshold value is small, the mask too becomes small, resulting in less of the targeted magnetite iron sands being identiﬁed. This in turn means none of the non- magnetite pixels become misidentiﬁed. However, where the threshold is larger, more magnetite iron sand pixels are identiﬁed. As a result, non-magnetite pixels too are likely to be misidentiﬁed as magnetite. This means that one has to manually adjust the threshold such that the mask ﬂawlessly covers the magnetite pixels only.
[page 9] Minerals 2022, 12, 268 9 of 19 Figure 5. SAM analysis at 2 m, 10 m and 20 m UAV drone elevations at different reference spectral thresholds. The red points represent the user-selected reference point known to be magnetite (pre- SAM), whilst the blue overlay represent areas within the threshold limit thought to be magnetite (post SAM cosine similarity). The best threshold cosine similarities are 0.12, 0.13 and 0.17 at 2 m, 10 m and 20 m, respectively. The x and y axis represent the size of each captured multispectral image. The bar on the right of each ﬁrst image represents the variation in spectral intensities present in each captured image. Reprinted with permission from ref. [1]. 2021 AusIMM [1] (p. 38).
[page 10] Minerals 2022, 12, 268 10 of 19 Based on Figure 5, the best SAM cosine similarity threshold values from visual inter- pretation were found to be 0.12, 0.13 and 0.17 for 2 m, 10 m and 20 m UAV ﬂight elevations, respectively. From these results, it can hence be said that distance from the sensor to the sub- ject affects the nature of the SAM analysis masks, given that SAM values required alteration for each ﬂight elevation. In summary, at lower UAV drone altitudes, a lower threshold was necessary to identify magnetite iron sands, whereas the opposite was correct at higher UAV drone altitudes. Parallax error was accounted for as previously described in Section 2.2 due to the multispectral sensors being located on different positions on the multispectral camera (Figure 2). This in turn results in a perfect overlay of all six multispectral images at each UAV ﬂight elevation. Additionally, with larger threshold values, more noise is introduced [16]. This noise is usually caused by mini-ripples and shadows. The SAM analysis is tricked by these ripples into assuming there is resemblance in image pixel characteristics between the actual reference subject and those expressed by ripples and shadows [16]. Even though a visual interpretation of the SAM mask maps seems somewhat instinctive, it can still be argued that this method provides valuable information pertaining to the location of a speciﬁc subject such as the magnetite iron sands. There is a plethora of applications within the mining industry where application of this UAV drone coupled with SAM analysis has the potential to be employed. Examples include resource exploration, resource mapping, resource estimation, mine site monitoring, and ore processing amongst others, where quick and accurate separation of rocks and/or minerals is frequently undertaken. 3.3. Application of AI Methods in Magnetite Spectral Classiﬁcation Post SAM AI algorithms are a trusted technique by which human error can be eliminated in an objective analysis of data [15], this in turn removes subjectivity pertaining to the visual interpretation of SAM segmentation maps. For each segmentation map, pixels within the 0.12, 0.13 and 0.17 cosine similarity thresholds for the 2 m, 10 m and 20 m ﬂight elevations, respectively, are labelled as magnetite. The rest of the pixel data that are outside the thresholds for each ﬂight elevation are labelled as non-magnetite. Figures 6 and 7 are representations of how the segmentation maps were extracted via SAM (shown as binarized images for easy visualization), hence creating the labelled segmentation maps across six spectral bands.

Spectral Angle Mapping and AI Methods Applied in Automatic Identification of Placer Deposit Magnetite Using Multispectral Camera Mounted on UAV


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant] Deep Q-learning is a deep reinforcement learning technique that extends the capabilities of clas sical Q-learning [34,35] by replacing the value iteration in the Q-table with the function approximator of deep artificial neural networks (ANN). Although applications of ANNs for geotechnical purposes are still often seen as complementary to conventional computational models [36,37], in this case the use of ANNs allows for applications in complex and continuous states spaces while classical Q-learning is confined to discrete states. DQN algorithms are off-policy, model free RL techniques following the Bellman equation (Eq. (1) after [14]) where the optimal action-value function Q*(s,a) is based on a state s and after having taken an action a. The best action is chosen by maximizing the expected value of r + γQ*(s′,a′) where r is the reward, γ is the discount factor that determines how important the future reward is to the algorithm (see also Table 2), s′ is the state at the next time step and a′ are all possible actions [14]. Q*(s, a) = Es′ [ r + γmax a′ Q*(s ′, a ′) |s, a ] (1) [14] have shown that the DQN algorithm can be used for a wide range of different RL problems and albeit there are improvements to deep Q- learning we chose the original implementation as we see it as well suited to establish a baseline for further developments in geotechnical RL. Our implementation is based on the DQN after [14] and the custom DQN implementation from [38]. The main deviations of the network archi tecture in comparison to [14,38] are due to the shape of the input and output data with the input being an 2 × 2100 × 2 array and the output a vector of length 8. The number of hidden layers and the decreasing kernel size from the top to the bottom convolutional layers is in accor dance with [14] as is the size of the kernel’s stride which is half the size of the kernel itself. While [14] used 32, 64 and 64 filters for each of the three hidden layers respectively, we used 32, 64 and 32 filters for these layers, as we observed that the agent’s performance did not suffer from this decrease, while the computational speed increased. In accordance with these authors we used rectified linear units (ReLU) [39] activation functions. ReLU activation functions have been widely adopted for ANNs within the past decade as they have shown to achieve a better performance than previously used activation functions like the sigmoid (see [40]). As given in [14] the general DQN’s ANN architecture is that of a deep convolutional neural network [41] whose hierarchical struc ture mimics the effect of receptive fields and is inspired by [42]. Like [14] we did not perform systematic hyperparameter tuning by random search or similar techniques (see e.g. [43]) due to the big computational effort of the simulation. Hyperparameters were thus optimized manually throughout the course of the development of the RL-simulation. We nevertheless point out that the given DQN architecture as well as the used hyperparameters still have room for improvement (see outlook in Section 7). Table 2 lists all hyperparameters for our DQN implementation. As we use the same terminology as [14] the reader is referred to this paper for more information on the individual parameters. We implemented the DQN using the tensorflow [44] based Python library Keras [45]. Training was done on a NVIDIA GeForce RTX 2080 Ti. From input to output, the agent’s architecture goes as follows and a graphical representation is given in Fig. 5 (with adaptions of architec ture from [14] as described in the section ahead): ▪ The input consists of an array with the shape 2 × 2100 × 2 (see Section 3.3.1). ▪ One convolutional layer, with 32 filters, a kernel size of 1 × 16 and a stride of 1 and 8 applying a ReLU activation function [39] ▪ One convolutional layer, with 64 filters, a kernel size of 1 × 8 and a stride of 1 and 4 applying a ReLU activation function ▪ One convolutional layer, with 32 filters, a kernel size of 1 × 4 and a stride of 1 and 2 applying a ReLU activation function ▪ One fully connected layer with 256 neurons applying a ReLU activation function ▪ One fully connected layer with 8 neurons (one per action) as the output layer which applies a linear activation Table 1 The mechanical parameters and permeability of the two ground types, where gt1 represents weak rock and gt2 stronger rock.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Automated Excavator Based on Reinforcement Learning and Multibody System Dynamics; DOI: 10.1109/access.2020.3040246; license: cc-by-sa.

[Assistant]
2016, doi: 10.1016/j.autcon.2016.05.009. [3] T. Hester, TEXPLORE: Temporal Difference Reinforcement Learning for Robots and Time-Constrained Domains, vol. 503. Cham, Switzerland: Springer, 2013, p. 164, doi: 10.1007/978-3-319-01168-4. [4] R. Sutton, F. Bach and A. Barto, Reinforcement Learning, 2nd ed. Cam- bridge, MA, USA: MIT Press, 2018, pp. 1–4. [5] X. Shi, P. J. A. Lever, and F.-Y. Wang, ‘‘Experimental robotic excavation with fuzzy logic and neural networks,’’ in Proc. IEEE Int. Conf. Robot. Autom., Apr. 1996, pp. 957–962, doi: 10.1109/robot.1996.503896. [6] A. Stentz, J. Bares, S. Singh, and P. Rowe, ‘‘Robotic excavator for autonomous truck loading,’’ Auto. Robots, vol. 7, no. 2, pp. 175–186, 1999, doi: 10.1023/A:1008914201877. [7] H. Shao, H. Yamamoto, Y. Sakaida, T. Yamaguchi, Y. Yanagisawa, and A. Nozue, ‘‘Automatic excavation planning of hydraulic excavator,’’ in Intelligent Robotics and Applications (Lecture Notes in Computer Sci- ence), vol. 5315. Berlin, Germany: Springer, 2008, pp. 1201–1211, doi: 10.1007/978-3-540-88518-4_128. [8] D. Schmidt, M. Proetzsch, and K. Berns, ‘‘Simulation and control of an autonomous bucket excavator for landscaping tasks,’’ in Proc. IEEE Int. Conf. Robot. Autom., May 2010, pp. 5108–5113, doi: 10.1109/ ROBOT.2010.5509546. [9] B. J. Hodel, ‘‘Learning to operate an excavator via policy optimization,’’ Procedia Comput. Sci., vol. 140, pp. 376–382, Jan. 2018, doi: 10.1016/j. procs.2018.10.301. [10] S. Dadhich, U. Bodin, F. Sandin, and U. Andersson, ‘‘Machine learn- ing approach to automatic bucket loading,’’ in Proc. 24th Medit. Conf. Control Autom. (MED), Jun. 2016, pp. 1260–1265, doi: 10.1109/MED. 2016.7535925. [11] Q. Zhu and Q.-F. Wang, ‘‘Real-time energy management controller design for a hybrid excavator using reinforcement learning,’’ J. Zhejiang Univ.-Sci. A, vol. 18, no. 11, pp. 855–870, Nov. 2017, doi: 10.1631/ jzus.A1600650. [12] J. Tobin, R. Fong, A. Ray, J. Schneider, W. Zaremba, and P. Abbeel, ‘‘Domain randomization for transferring deep neural networks from simu- lation to the real world,’’ in Proc. IEEE/RSJ Int. Conf. Intell. Robots Syst. (IROS), 2017. Accessed: Nov. 25, 2020, doi: 10.1109/iros.2017.8202133. [13] J. Xu, T. Du, M. Foshey, B. Li, B. Zhu, A. Schulz, and W. Matusik, ‘‘Learning to ﬂy?: Computational controller design for hybrid UAVs with reinforcement learning learning to ﬂy?: Computational controller design for hybrid UAVs with reinforcement learning,’’ ACM Trans. Graph., vol. 38, no. 4, pp. 1–12, Jul. 2019, doi: 10.1145/3306346.3322940. [14] Z. Xie, P. Clary, J. Dao, P. Morais, J. Hurst, and M. Van De Panne, ‘‘Iterative reinforcement learning based design of dynamic locomo- tion skills for cassie,’’ 2019, arXiv:1903.09537. [Online]. Available: https://arxiv.org/abs/1903.09537 [15] P. Hämäläinen, A. Babadi, X. Ma, and J. Lehtinen, ‘‘PPO-CMA: Prox- imal policy optimization with covariance matrix adaptation,’’ 2018, arXiv:1810.02541. [Online]. Available: http://arxiv.org/abs/1810.02541 [16] J. de García Jalón, E. Álvarez, F. A. de Ribera, I. Rodríguez, and F. J. Funes, ‘‘A fast and simple semi-recursive formulation for multi-rigid- body systems,’’ in Advances in Computational Multibody Systems. Dor- drecht, The Netherlands: Springer, 2005, pp. 1–23. VOLUME 8, 2020 214005
[page 9] I. Kurinov et al.: Automated Excavator Based on RL and Multibody System Dynamics [17] A. Avello, J. M. Jiménez, E. Bayo, and J. G. de Jalón, ‘‘A simple and highly parallelizable method for real-time dynamic simulation based on velocity transformations,’’ Comput. Methods Appl. Mech. Eng., vol. 107, no. 3, pp. 313–339, Aug. 1993, doi: 10.1016/0045-7825(93)90072-6. [18] J. G. de Jalón and E. Bayo, Kinematic and Dynamic Simulation of Multi- body Systems: The Real-Time Challenge. New York, NY, USA: Springer, 1993. [19] Y. Pan, W. Dai, Y. Xiong, S. Xiang, and A. Mikkola, ‘‘Tree-topology- oriented modeling for the real-time simulation of sedan vehicle dynam- ics using independent coordinates and the rod-removal technique,’’ Mechanism Mach. Theory, vol. 143, Jan. 2020, Art. no. 103626, doi: 10.1016/j.mechmachtheory.2019.103626. [20] J. Watton, Fluid Power Systems?: Modeling, Simulation, Analog and Microcomputer Control. Upper Saddle River, NJ, USA: Prentice-Hall, 1989. [21] H. M. Handroos and M. J. Vilenius, ‘‘Flexible semi-empirical models for hydraulic ﬂow control valves,’’ J. Mech. Design, vol. 113, no. 3, pp. 232–238, Sep. 1991, doi: 10.1115/1.2912774. [22] M. Pla-Castells, I.

Automated Excavator Based on Reinforcement Learning and Multibody System Dynamics


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Detection of Ionospheric Scintillation Based on XGBoost Model Improved by SMOTE-ENN Technique; DOI: 10.3390/rs13132577; license: cc-by-sa.

[Assistant]
Resampling Technique Raw Data Random Oversampling Random Undersampling SMOTE ENN SMOTE-ENN Precision (%) 0 99.38 99.51 99.64 99.77 99.47 99.71 1 82.45 83.07 85.00 85.05 86.67 87.96 2 96.02 95.14 94.83 95.19 95.77 95.00 Recall (%) 0 87.91 87.55 89.43 89.31 92.13 92.16 1 97.89 97.64 97.65 97.93 97.85 97.77 2 86.55 90.15 90.75 91.17 87.47 91.60 F1 score (%) 0 93.29 93.15 94.26 94.25 95.66 95.79 1 89.51 89.77 90.89 91.04 91.93 92.61 2 91.01 92.58 92.75 93.14 91.43 93.27 Accuracy (%) 91.43 91.69 92.69 92.80 93.58 94.17 Based on the SMOTE-ENN resampling technique, Table 5 compares the experimental data for ﬁve detection algorithms. The decision tree and random forest, each of which presented excellent performance in the binary classiﬁcation for ionospheric scintillation detection in recent research [16], were tested for comparison. Considering the good per- formance of the SVM in similar detection tasks [19–23], tests for such a method were performed and analyzed. The CNN model was also employed in the comparison due to its effectiveness in a wide range of classiﬁcation problems [25–28]. However, the SMOTE-ENN resampling technique is meaningless in regard to improving CNN performance, as its space structure becomes disordered after the resampling process. Thus, the CNN test was carried out on raw data only, while the same tests on the SVM accord with those on the other three methods. Due to the mentioned problem of the CNN and excessive computational loads of the SVM, these two methods were not studied further. As shown in
[page 14] Remote Sens. 2021, 13, 2577 14 of 22 Table 5, apart from the overall accuracy, the recall and F1 score for class 2 were recorded to illustrate the performance in detecting strong scintillation events. In addition, considering the unavailability of the resampling technique for the CNN, larger computational loads and running time for the SVM, and the overall better performance with XGBoost, further tests with both methods were not carried out. Further details can be found in the Discussion. Table 5. Comparison of performance of different detection algorithms trained with raw dataset and SMOTE-ENN resampled dataset (apart from CNN algorithm), based on testing results. Algorithm Accuracy (%) Recall for Class 2 (%) F1 Score for Class 2 (%) Raw Data SMOTE- ENN Improvement Ratio Raw Data SMOTE- ENN Improvement Ratio Raw Data SMOTE- ENN Improvement Ratio CNN 89.86 - - 73.63 - - 82.15 - - SVM 93.39 93.52 0.13 90.49 95.06 4.57 89.46 89.07 -0.39 Decision tree 85.39 87.36 1.97 86.73 90.41 3.68 88.94 90.98 2.04 Random forest 86.95 88.92 1.97 85.44 89.97 4.53 90.43 92.91 2.48 XGBoost 91.43 94.17 2.74 86.55 91.60 5.05 91.04 93.27 2.23 When training on the raw data and resampled data, the accuracy of XGBoost was about 4% and 7% better than that of the decision tree and random forest, as shown in the left panel of Figure 8. The middle and right panels in Figure 8 also show higher recall and F1 scores for the XGBoost method, implying the superiority of XGBoost improved by the SMOTE-ENN technique. 85 87 89 91 93 95 Raw data SMOTE-ENN 2.74% growth (a) 86 88 90 92 Raw data SMOTE-ENN (b) 5.05% growth 89 90 91 92 93 Raw data SMOTE-ENN 2.23% growth (c) Figure 8. Comparison of detection algorithms in terms of overall accuracy, recall value for class 2 and F1 scores for class 2, respectively. From left to right: (a) the overall accuracy; (b) the recall value for class 2; (c) the F1 score for class 2. The blue polyline marked by triangles refers to the testing results based on raw training data, while the orange polyline marked by asterisks represents the testing results based on SMOTE-ENN resampled training data. The 100,000 training samples were selected in the overall training dataset listed in Table 1, while the testing dataset was that mentioned in Table 1. 4.4. Analysis on Imbalanced Training Datasets To explore the detection performance of the XGBoost algorithm improved by the SMOTE-ENN resampling technique for strong scintillation events, a series of comparison experiments were performed based on different degrees of imbalance in the training and testing datasets, compared with the decision tree and random forest algorithm. From the training dataset with 349,559 samples, seven subsets of data were extracted. For each subset, 100,000 randomly selected samples were included, with the ratio of classes 0, 1 and 2 ranging from 1:1:1 to 7:7:1, respectively.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Reinforcement learning based process optimization and strategy development in conventional tunneling; DOI: 10.1016/j.autcon.2021.103701; license: cc-by.

[Assistant]
A solution would be to take a version of the agent that was saved before instability occurred and continue training with that, but we left Fig. 7. Different training paths for five exemplary agents over 120,000 episodes. The first row shows cumulative rewards per episode; the second row the average number of blasts/moves that were required to complete each episode; the third row the average number of face instabilities per episode; the fourth row shows the average loss per episode. Transparent colors in the background show the raw data records and the solid lines in the foreground a 500-episode sliding window average. Fig. 8. Histogram for the performance of an agent who plays 10,000 episodes with completely random moves (i.e. ε = 1). G.H. Erharter et al.
[page 9] Automation in Construction 127 (2021) 103701 9 the agents 2 and 5 as they are for explanatory purposes. Before discussing the performance of the agents, we first establish a baseline for the minimum performance that an agent must reach by letting an agent play with completely random moves (i.e. ε = 1) for 10,000 episodes. In these 10,000 random episodes the agent reached a ▪ minimum/maximum/median reward of −544/255/-13 points, ▪ minimum/maximum/median number of 0/27/8 instabilities per episode, ▪ minimum/maximum/median number of 124/197/145 blasts/ moves per episode. Histograms for the 10,000 random episodes are given in Fig. 8. The maximum rewards, minimum number of face instabilities and minimum number of blasts/moves that the five agents of Fig. 7 needed are given in Table 5. We computed these numbers based on sliding window averages of 500 episodes as to avoid individual episodes that performed extraordinarily well. Albeit substantial differences can be seen in these statistics, it can generally be observed that all maximum rewards of the agents are above the maximum rewards of the randomly played episodes and therefore training was generally a success. In these five agents, two groups can be observed, where the agents 2 and 3 reached their maximum performance between episodes 40,000 and 60,000 and the agents 1, 4 and 5 reached their maximum perfor mance between episodes 80,000 and 120,000. Although the first group has reached their peak performance sooner, the achieved reward is generally lower than that of the second group (see description of training paths above). We conclude this section by presenting the strategy that the best performing agent 4 has found after 119,000 episodes – at its peak per formance (in terms of highest reward and low instabilities). We tested the saved agent’s checkpoint for 10,000 episodes with a fixed ε of 0.05 which corresponds to the ε at that stage of training and is in accordance to [14] who recommend an ε > 0 also for testing, as this helps the agent to deal with unexpected situations. The histograms of Fig. 9 show the same test statistics as given for the random moves in Fig. 8. The strategy that the agent adopted is focused on long advance lengths without face support, in alternation with long advance lengths with face support. By doing so the agent avoids face instabilities, while also minimizing the required support (see Table 4 for the respective rewards/penalties). The boxplot of Fig. 10 illustrates this, as it can be observed that the majority of actions is 4 m long advance lengths. Furthermore, the agent focuses on long advance lengths without face support in both excavation types (top heading and bench) which shows that it tries to avoid excessive use of support measures, thus showing a tendency towards economical optimization. It can also be seen that the agent still uses small advance lengths sometimes and, in this case, favors the actions without face support as it has realized that the small advance lengths do not lead to face instabilities in the given conditions. In Fig. 11 an example of one episode for the agent 4 is visualized. The time-distance diagram in the top row of this figure shows that the agent has learned to optimize the excavation process by minimizing changes between top heading and bench excavation which would be associated with unwanted delays in “real life” tunneling (e.g. building and removing of access ramps). As given in Section 2.1, installed support ahead of the current face always covers 10 m. With the maximum advance length being set to 4 m it would be unnecessary and uneco nomical to install face support in two consecutive rounds. In the second row of Fig.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Intelligent Classification of Surrounding Rock of Tunnel Based on 10 Machine Learning Algorithms; DOI: 10.3390/app12052656; license: cc-by-sa.

[Assistant]
The tunnel is 5498 m in length with a maximum buried depth of 320 m. The main lithologies of the surrounding rock are limestone and shale, and the grades of the surrounding rocks are III (1493 m), IV (1050 m), and V (720 m). The method of construction of this tunnel is the full-section method with large ma- chinery, i.e., the tunnel face (150 m2) is excavated and formed by blasting once, and the primary support is closed once. With this method, the circular footage is 2–4.8 m, and the monthly progress is 60–150 m in this tunnel. The tunnel position is shown in Figure 12. This test section is DK450 + 834~DK451 + 126, which is 292 m in total length. The excavation revealed that the surrounding rock was grade-III; the lithology was bluish grey, a massive structure, and hard limestone rock. The stability of the tunnel face was good without developed joints, fractures, or groundwater. The test section of this tunnel is shown in Figure 13, and the tunnel face is shown in Figure 14.
[page 16] Appl. Sci. 2022, 12, 2656 16 of 20 Figure 12. Interface of the position of the Gaojiaping Tunnel of the Zhengzhou–Wanzhou high- speed railway. DK451+126 Tunnel Entrance mileage:DK449+037 DK450+834 DK451+000 Limestone(T1d) Exit mileage:DK454+535 Wanzhou Zhenzhou 650 450 250 500 300 H（m） 550 350 600 400 200 Figure 13. Test section of Gaojiaping Tunnel.
[page 17] Appl. Sci. 2022, 12, 2656 17 of 20 Figure 14. Tunnel face of this test section of the Gaojiaping Tunnel. 3.3. Intelligent Surrounding Rock Classiﬁcation in Test Tunnel Among the test section, drilling parameter information from 30 tunnel faces was col- lected by the intelligent surrounding rock classiﬁcation system. The intelligent classiﬁcation results of the surrounding rock on each tunnel face are shown in Table 8. Table 8. Intelligent classiﬁcation of surrounding rock on each tunnel face. Number Mileage Predicted Results Actual Results Right/Error 1 DK450+834 III III Right 2 DK450+837 III III Right 3 DK450+840 III III Right 4 DK450+916 III III Right 5 DK450+919 III III Right 6 DK450+922 III III Right 7 DK450+926 III III Right 8 DK451+014 III III Right 9 DK451+017 III III Right 10 DK451+021 III III Right 11 DK451+024 III III Right 12 DK451+027 III III Right 13 DK451+030 III III Right 14 DK451+034 III III Right 15 DK451+037 III III Right 16 DK451+042 III III Right 17 DK451+046 III III Right 18 DK451+051 III III Right 19 DK451+055 III III Right 20 DK451+060 III III Right 21 DK451+064 III III Right 22 DK451+100 III III Right 23 DK451+103 III III Right 24 DK451+106 III III Right 25 DK451+109 III III Right 26 DK451+113 IV III Error 27 DK451+116 IV III Error 28 DK451+119 IV III Error 29 DK451+123 IV III Error 30 DK451+126 III III Right Accuracy 86.7%
[page 18] Appl. Sci. 2022, 12, 2656 18 of 20 In Table 8, the classiﬁcation accuracy of the surrounding rock intelligent classiﬁcation system on-site is 86.7%, which indicates that the system has good generalization. However, due to the single lithology and surrounding rocks (limestone, grade-III) in the ﬁeld test section, its generalization performance must be further veriﬁed. 4. Conclusions Based on 912 drilling parameters of the Zhengzhou–Wanzhou high-speed railway tunnel project, 10 intelligent surrounding rock classiﬁcation models were established using multiple machine learning algorithms. With three models (SVM, RT, and ET), an intelligent surrounding rock classiﬁcation system was established and veriﬁed by the ﬁeld test section. The main conclusions of this study are: 1. All the absolute values of the correlation coefﬁcients between the four drilling pa- rameters (Vp, Ph, Pr, and Pf) and the surrounding rock grades (III, IV, and V) are above 0.5. Thus, there is a correlation between the drilling parameters and the surrounding rock grade, so the method of surrounding rock classiﬁcation by drilling parameters is feasible. 2. The 10 models based on supervised machine learning algorithms all have good performance. The average accuracy of them is 0.82. In particular, the average recalls of grades-III and -V were greater than 0.90. Thus, these machine learning models established by drilling parameters are feasible and reliable in the intelligent classiﬁcation of surrounding rocks. 3. By comprehensively considering precision, recall, F1 score, and accuracy, we observe that three types of models containing SVM, RT, and ET have better performance among 10 machine learning algorithm models.


[System] You are Gemma4Tunnel, a careful tunnel boring machine and tunnelling engineering assistant. Use engineering terminology precisely, state assumptions, and avoid inventing source claims.

[User]
Extract reusable tunnelling/TBM engineering knowledge from this allowed source excerpt. Preserve important parameters, mechanisms, risks, and caveats. Do not add unsupported claims.

Source: Wheel Loader Scooping Controller Using Deep Reinforcement Learning; DOI: 10.1109/access.2021.3056625; license: cc-by-sa.

[Assistant]
[page 1] Received January 12, 2021, accepted January 20, 2021, date of publication February 2, 2021, date of current version February 10, 2021. Digital Object Identifier 10.1109/ACCESS.2021.3056625 Wheel Loader Scooping Controller Using Deep Reinforcement Learning OSHER AZULAY 1 AND AMIR SHAPIRO 2, (Member, IEEE) 1School of Mechanical Engineering, Tel Aviv University, Tel Aviv 69978, Israel 2Department of Mechanical Engineering, Ben-Gurion University of the Negev, Be’er Sheva 84105, Israel Corresponding author: Osher Azulay (osherazulay@mail.tau.ac.il) This work was supported by the Israeli Ministry of Defense through Project ROBIL under Grant 87726311. ABSTRACT This article presents a deep reinforcement learning-based controller for an unmanned ground vehicle with a custom-built scooping mechanism. The robot’s aim is to autonomously perform earth scooping cycles with three degrees of freedom: lift, tilt and the robot’s velocity. While the majority of previous studies on automated scooping processes are based on data recorded by expert operators, we present a method to autonomously control a wheel loader to perform the scooping cycle using deep reinforcement learning methods without any user-provided demonstrations. The controller’s learning approach is based on the actor- critic, Deep Deterministic Policy Gradient algorithm which we use to map online sensor data as input to continuously update the actuator commands. The training of the scooping policy network is done solely in a simpliﬁed simulation environment using a virtual physics engine, which converges to an average of a 65% ﬁll factor from the full bucket capacity and a 5 [sec] average cycle time. We illustrate the performance of the trained policy in simulations and in real-world experiments with 3 different inclination angles of the earth. An additional scooping experiment compared the performance of our controller to remote manual human control. Overall, the deep reinforcement learning-based controller exhibited good performance in terms of both achieved visually bucket ﬁll with varying scooped earth weights of 4.1 −7.2[kg], and a 5.1 −7.1[sec] cycle time. The experimental results conﬁrm the ability of our planner to ﬁll bucket as required, indicating that our controller can be used for excavation purposes. INDEX TERMS Robotics in construction, machine learning, agricultural automation. I. INTRODUCTION Earthmoving systems are currently used in a variety of industries, but especially in the construction and agricul- tural domains. They have numerous advantages including economic efﬁciency, safety, and availability. Earthmoving machinery typically refers to heavy-duty vehicles designed for construction operations that involve earthworks. While earthmoving machinery continues to develop, most of the excavation cycle is still controlled by a human, either directly or via teleoperation [1]. Designing control methods for such tasks is a long-standing research goal, which has attracted considerable interest and generated a number of survey papers [2], [3]. The general approaches to autonomous excavation exploit the machine dynamics and try to follow a deﬁned trajectory [4], [5], use compliance force control [6], [7] or employ a behavior-based approach for motion The associate editor coordinating the review of this manuscript and approving it for publication was Yangmin Li . FIGURE 1. A deep reinforcement learning controller was trained in simulation to accomplish the scooping cycle using 12-dimensional sensors inputs into three actuator commands. control [8], [9]. Since some tasks are more complex than oth- ers, they often require extensive engineering experience and tedious manual tuning beyond the control algorithm itself. One of the key challenges in deploying automated earth moving machines relates to the analysis of the soil-tool interaction, due to the unpredictable nature of the soil [6]. VOLUME 9, 2021 This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/ 24145
[page 2] O. Azulay, A. Shapiro: Wheel Loader Scooping Controller Using Deep Reinforcement Learning Most approaches require accurate models of the machine which makes them liable to modeling errors, wear and tear, and changing conditions [10].