Markov Decision Process (MDP) models are widely used to model decision-making problems in many research fields. MDPs can be readily designed through modeling and simulation(M&S) using the Discrete Event System Specification formalism (DEVS) due to its modular and hierarchical aspects, which improve the explainability of the models. In particular, the separation between the agent and the environment components involved in the traditional reinforcement learning (RL) algorithm, such as Q-Learning, is clearly formalized to enhance observability and envision the integration of AI components in the...