Academia.eduAcademia.edu

Figure 7 – uploaded by Elpiniki Papageorgiou

See full PDFdownloadDownload figure
these relationships are only activated when its respective states are triggered, e.g., W, represents the three causal relationships between the input concepts (CLVP, CFVP or CRVP) and the escape state (concept C,). The weights W. and Ws represent the relationships between the inputs and {Cs, Co} when the search for feed state (in the graph as Selection: feed) is triggered; similarly for W; and Wz. when the obstacle deviation state is triggered. The difference between these states is the influence of the inputs concepts into the output concepts, when selection “feed” is activated, the concepts C; and C; infer the concept Cs; and C, respectively; when the selection “obstacle” is triggered the concepts C, and C; infer C, and C; respectively. In other words, if the agent perceives an obstacle at its right, it will react by turning left; if the object perceived is a feed instead of an obstacle, the agent will turn right, approaching the feed. This can also be seen in its respective tuples, represented by the equations 9 to 14 to be presented below. According to section 2.3, the weights are described by their tuples, as following to W, and W;:

Figure 7 these relationships are only activated when its respective states are triggered, e.g., W, represents the three causal relationships between the input concepts (CLVP, CFVP or CRVP) and the escape state (concept C,). The weights W. and Ws represent the relationships between the inputs and {Cs, Co} when the search for feed state (in the graph as Selection: feed) is triggered; similarly for W; and Wz. when the obstacle deviation state is triggered. The difference between these states is the influence of the inputs concepts into the output concepts, when selection “feed” is activated, the concepts C; and C; infer the concept Cs; and C, respectively; when the selection “obstacle” is triggered the concepts C, and C; infer C, and C; respectively. In other words, if the agent perceives an obstacle at its right, it will react by turning left; if the object perceived is a feed instead of an obstacle, the agent will turn right, approaching the feed. This can also be seen in its respective tuples, represented by the equations 9 to 14 to be presented below. According to section 2.3, the weights are described by their tuples, as following to W, and W;:

Related Figures (23)

For use in artificial life, a reinforcement learning algorithm similar to the Q-learning and based on _ heuristic is implemented. The standard RL algorithm uses a policy to map states and actions, i.e., the policy determine what at action should be performed when agent is in a s, state, thus, defining the agent behavior over time. This algorithm uses the experience of each state transition to update the cost Q(s, a) of taking an action a that drives the system from the actual state s, to next state s,,;.. This up-dated is carried out by means of the rule: One of the advantages of this research, compared to the original work [12], is to include the self-adaptation capacity to the agents through the use of Reinforcement Learning (RL) for dynamic tuning. A RL algorithm allows adjustments for a better adaptation through refinement of DFCM weights, due possible changes in the environment and / or the dynamics of the agent or creature (e.g., worn out parts); i.e., the agent interacts with the environment through the RL dynamical sets as shown in figure 1.
For use in artificial life, a reinforcement learning algorithm similar to the Q-learning and based on _ heuristic is implemented. The standard RL algorithm uses a policy to map states and actions, i.e., the policy determine what at action should be performed when agent is in a s, state, thus, defining the agent behavior over time. This algorithm uses the experience of each state transition to update the cost Q(s, a) of taking an action a that drives the system from the actual state s, to next state s,,;.. This up-dated is carried out by means of the rule: One of the advantages of this research, compared to the original work [12], is to include the self-adaptation capacity to the agents through the use of Reinforcement Learning (RL) for dynamic tuning. A RL algorithm allows adjustments for a better adaptation through refinement of DFCM weights, due possible changes in the environment and / or the dynamics of the agent or creature (e.g., worn out parts); i.e., the agent interacts with the environment through the RL dynamical sets as shown in figure 1.
Fig. 3. — Dolphin - Hierarchical decisions (Bottom-up) and adaptive layers
Fig. 3. — Dolphin - Hierarchical decisions (Bottom-up) and adaptive layers
This architecture exploits the structured knowledg cquisition capacity, with different sources of knowledge uch sources may or may not have the same origin, whic ermits the construction of low-level decisions-making (caus: elations) and strategic. There is a direct mapping of th ituation to action which is arranged in hierarchical layers. I eneral, the different layers are not completely independen ‘he system may be able to handle more than one task i arallel, despite its dependence. Consequently, ubsumption architecture provides mechanisms igher or competent layers can observe the state of mes, which may inhibit their outputs and their in th by whic the lowe puts, thu: djusting their behavior. The DFCM architecture wa riginally designed for robots (agents) whose contrc¢ nechanisms are not centralized. Figure 3 s NOWS th ierarchical actions layers proposed for a DFCM topology to olphin and Figure 4 presents the hierarchical layers to redator. -——_————_
This architecture exploits the structured knowledg cquisition capacity, with different sources of knowledge uch sources may or may not have the same origin, whic ermits the construction of low-level decisions-making (caus: elations) and strategic. There is a direct mapping of th ituation to action which is arranged in hierarchical layers. I eneral, the different layers are not completely independen ‘he system may be able to handle more than one task i arallel, despite its dependence. Consequently, ubsumption architecture provides mechanisms igher or competent layers can observe the state of mes, which may inhibit their outputs and their in th by whic the lowe puts, thu: djusting their behavior. The DFCM architecture wa riginally designed for robots (agents) whose contrc¢ nechanisms are not centralized. Figure 3 s NOWS th ierarchical actions layers proposed for a DFCM topology to olphin and Figure 4 presents the hierarchical layers to redator. -——_————_
Fig. 5. — Dolphin’s DFCM- Subsumption strategy modeled as states machine.
Fig. 5. — Dolphin’s DFCM- Subsumption strategy modeled as states machine.
Fig. 6. — Predator’s DFCM-Subsumption strategy modeled as states machine.
Fig. 6. — Predator’s DFCM-Subsumption strategy modeled as states machine.
Fig. 8. -DFCM resulting from obstacle deviation selection One example of these permutations of the DFCM can be seen in Figure 8, where only the obstacle deviation state has been triggered. There are no visual representation of weights between the input concepts (C;, C, and C;) and the selections (Ws; and Ws,4), due to a visual simplification in the cognitive model to facilitate the understanding of the DFCM’s graph. In resume, figure 8 presents the resulting DFCM graph to the obstacle deviation state. It serves as a_ general exemplification of its functioning, related to the actuation of the selection process. In the obstacle avoidance’s state, while there is no detection of feed, predator or same species’ animals, the causal relationships related to these concepts are null, i.e., these concepts do not influence the decision-making process. However, the cognitive model can change to another state (according to the states machine), rearranging itself.
Fig. 8. -DFCM resulting from obstacle deviation selection One example of these permutations of the DFCM can be seen in Figure 8, where only the obstacle deviation state has been triggered. There are no visual representation of weights between the input concepts (C;, C, and C;) and the selections (Ws; and Ws,4), due to a visual simplification in the cognitive model to facilitate the understanding of the DFCM’s graph. In resume, figure 8 presents the resulting DFCM graph to the obstacle deviation state. It serves as a_ general exemplification of its functioning, related to the actuation of the selection process. In the obstacle avoidance’s state, while there is no detection of feed, predator or same species’ animals, the causal relationships related to these concepts are null, i.e., these concepts do not influence the decision-making process. However, the cognitive model can change to another state (according to the states machine), rearranging itself.
Once the DFCM models (valid set of weights and concepts in each time instant) to dolphins and sharks are defined by the internal state system, the weights can be finely tuned by the reinforcement learning algorithm. This adaptation of DFCM models must be carefully designed to avoid the occurrence of drastic changes in the values of the weights. Figure 9 shows the predator’s DFCM. Its functioning is similar to the dolphin’s DFCM, in a simplified way. It gives the predator ability to avoid obstacles, search for feed (hunt the prey, in this virtual environment). This model is in accordance with predator state machine strategy. Where PLVP (Predator Left Visual Perception), PFVP (Predator Frontal Visual Perception) and PRVP (Predator Right Visual Perception) are the input concepts, and TL (Turn Left) and TR (Turn Right) are output concepts. The causal and selection relationships are similar to Dolphin DFCM, as its finite state machine triggering its related concepts.
Once the DFCM models (valid set of weights and concepts in each time instant) to dolphins and sharks are defined by the internal state system, the weights can be finely tuned by the reinforcement learning algorithm. This adaptation of DFCM models must be carefully designed to avoid the occurrence of drastic changes in the values of the weights. Figure 9 shows the predator’s DFCM. Its functioning is similar to the dolphin’s DFCM, in a simplified way. It gives the predator ability to avoid obstacles, search for feed (hunt the prey, in this virtual environment). This model is in accordance with predator state machine strategy. Where PLVP (Predator Left Visual Perception), PFVP (Predator Frontal Visual Perception) and PRVP (Predator Right Visual Perception) are the input concepts, and TL (Turn Left) and TR (Turn Right) are output concepts. The causal and selection relationships are similar to Dolphin DFCM, as its finite state machine triggering its related concepts.
Fig. 12. — Zoomed the virtual creatures” trails when they see each other.
Fig. 12. — Zoomed the virtual creatures” trails when they see each other.
Fig. 13. — Zoomed trails of dolphin in a grouping moment
Fig. 13. — Zoomed trails of dolphin in a grouping moment
Fig. 14. — Dolphin’s acceleration (Concept Cs) and relative distance between the agents — environment with one predator. (central plot) and right (bottom plot) perception measurements are similar, indicating dolphin closeness to these obstacles.
Fig. 14. — Dolphin’s acceleration (Concept Cs) and relative distance between the agents — environment with one predator. (central plot) and right (bottom plot) perception measurements are similar, indicating dolphin closeness to these obstacles.
Fig. 16. — Dolphin’s decision about Turn Left (Cs) and Turn Right (Ce) - environment with one predator. Fig. 15. — Dolphin’s perception measurements (input concepts) and weights - environment with one predator.
Fig. 16. — Dolphin’s decision about Turn Left (Cs) and Turn Right (Ce) - environment with one predator. Fig. 15. — Dolphin’s perception measurements (input concepts) and weights - environment with one predator.
Fig. 17. —Dolphin’s perception measurements (input concepts) and weights during escape from predator — environment with one predator.
Fig. 17. —Dolphin’s perception measurements (input concepts) and weights during escape from predator — environment with one predator.
Fig. 18. —Virtual creature’s trail in the environment, when the predator catches its prey. The Cognitive Model of the shark is relatively simple, with only two states, avoid obstacles and hunt a prey. The state change occurs in case of prey (dolphin) perception; thus the predator enters in the persecution state (hunt) when its speed is increased according to its hunger. Figure 18 shows the creatures” trace in the simulated virtual environment to a new >xperiment, in which the predator captures its prey. The oredator’s concept of "hunger" had an increase of 30% than he previous simulation (figure 11), thus allowing the predator o hunt and capture its prey successfully. This can be seen in figures 19 to 21, where the dolphin perception ceases about 320 cycles when the dolphin is eaten by the predator: distance between both creatures is null in the bottom plot of figure 19 and all concepts (C; to C; in figure 20, Cs and C¢ in figure 21 and Cx in figure 19) are also null.
Fig. 18. —Virtual creature’s trail in the environment, when the predator catches its prey. The Cognitive Model of the shark is relatively simple, with only two states, avoid obstacles and hunt a prey. The state change occurs in case of prey (dolphin) perception; thus the predator enters in the persecution state (hunt) when its speed is increased according to its hunger. Figure 18 shows the creatures” trace in the simulated virtual environment to a new >xperiment, in which the predator captures its prey. The oredator’s concept of "hunger" had an increase of 30% than he previous simulation (figure 11), thus allowing the predator o hunt and capture its prey successfully. This can be seen in figures 19 to 21, where the dolphin perception ceases about 320 cycles when the dolphin is eaten by the predator: distance between both creatures is null in the bottom plot of figure 19 and all concepts (C; to C; in figure 20, Cs and C¢ in figure 21 and Cx in figure 19) are also null.
Fig. 19. — Dolphin’s acceleration (Concept Cg) and relative distance between the agents during dolphin’s capture and death.
Fig. 19. — Dolphin’s acceleration (Concept Cg) and relative distance between the agents during dolphin’s capture and death.
Fig. 20. — Dolphin’s perception measurements (input concepts) and weights during its capture and death.
Fig. 20. — Dolphin’s perception measurements (input concepts) and weights during its capture and death.
Fig. 21.— Dolphin’s decision about Turn Left (Cs) and Turn Right (C,) during its capture and death.
Fig. 21.— Dolphin’s decision about Turn Left (Cs) and Turn Right (C,) during its capture and death.
ig. 23. — Dolphin’s acceleration (Concept Cs) and relative distance between the agents — environment with two predators. Fig. 22. —Virtual creatures” trail in an environment with two predators.
ig. 23. — Dolphin’s acceleration (Concept Cs) and relative distance between the agents — environment with two predators. Fig. 22. —Virtual creatures” trail in an environment with two predators.
Fig. 24. — Dolphin’s perception measurements (input concepts) and weights— environment with two predators. Figures 24 shows the values of the dolphin’s perception system that correspond to the DFCM input concepts related to the presence of obstacles, feed, predator and another dolphin. This figure also shows the weights related to obstacle deviation W3 and W, that are tuned by reinforcing learning algorithm. Figure 25 presents the actions to turn left (concept Cs) due to the perception of an obstacle (red line) in the opposite side or food (blue line) in the same side.
Fig. 24. — Dolphin’s perception measurements (input concepts) and weights— environment with two predators. Figures 24 shows the values of the dolphin’s perception system that correspond to the DFCM input concepts related to the presence of obstacles, feed, predator and another dolphin. This figure also shows the weights related to obstacle deviation W3 and W, that are tuned by reinforcing learning algorithm. Figure 25 presents the actions to turn left (concept Cs) due to the perception of an obstacle (red line) in the opposite side or food (blue line) in the same side.
hunger. However, when the predator has a slower speed, the dolphin is able to escape. Fig. 25. — Dolphin’s decision about Turn Left (Cs) and Turn Right (C,) - environment with two predators.
hunger. However, when the predator has a slower speed, the dolphin is able to escape. Fig. 25. — Dolphin’s decision about Turn Left (Cs) and Turn Right (C,) - environment with two predators.
Related topics:
Computer ScienceArtificial IntelligenceReinforcement LearningArtificial LifeFuzzy Cognitive Map
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved