Figure 5 14 Agent states of punishment sensitivity statechart
Related Figures (82)
Fig. 2.1 Outline of the Background Chapter his chapter presents an introduction to the fields of Game Theory, Experimental and Be avioural Economics in Section 2.1, and Agent-Based Modelling and Simulation in Section 2.2 hen Section 2.3 discusses Public Goods Game and overview of three research approaches amely Game Theory, Laboratory Experiment, and Agent-Based Modelling and Simulation .fter discussing the literate streams, the research gap is derived in Section 2.4. Figure 2.1 ummarizes the outline of this chapter. Consider the prisoner’s dilemma from perspective of prisoner A. He has to make this choic yased on what he believes B will choose. If B denies, A’s best response is to confess sinc 1e will go free compared with | year jail. But if B confesses, A’s best response 1s to confes 1e will get 2 years instead of 3. Therefore, Confess is the best strategy for A regardless c 1is belief about B’s choice. Confess is called a dominant strategy and Deny is a dominate ‘trategy. Since prisoner B is playing the same game, Confess is also the dominant strategy fc 1im. Therefore, the final result is that both will confess and get 3-year jail sentences. Eve hough they can both deny and get just 1 year each, they do not do it because the choice (Den Deny) is unstable, which means that the other prisoner can get a better payoff by changin rom Deny to Confess. The state that both prisoners confess is called Nash Equilibrium, whic Fig. 2.2 The interactions between MAS, ABMS, ACE 2.2 Agent-Based Modelling and Simulation Table 2.5 Variables and their effects on contribution, adapted from Ledyard (1995) Fig. 3.1 Research activities Fig. 3.4 The lifecycle of a simulation study (Robinson, 2004) Fig. 3.5 Overview lifecycle of the ABOOMS framework v0 3.5 Design of the Initial Framework Fig. 3.6 An object-oriented development process 3.5 Design of the Initial Framework 3.5 Design of the Initial Framework he first part contains the name of the class; the second part the attributes of the class; and the urd part the methods or operations that the class can perform. Figure 3.8 is an example o: ‘ustomer class of the Bank ATM example. The Customer class has three attributes (id, name alance) and can perform three operations (checkBalance, deposit, withdraw). Each attribute as the type listed right after its name, separated by a colon (id, name, balance have types of! tring, String, double). Branches must be deleted if the Bank does not exist. the transitions between them, and can be used to visualize and model the reaction of objects by Figure 3.10 shows the state machine of an example Customer class, which has two mait tates: “Idle”, and “Using ATM”. In the “Using ATM” state, there are four sub-states: “Auther ication’, “Checking balance”, “Deposit”, “Withdraw”. At the top left is the initial state, whicl nitializes this diagram to be “Idle”. When a user starts to use the ATM, transition from “Idle” t Using ATM” will be triggered. When in “Using ATM”, the initial sub-sta f the user fails to authenticate, a self-transition will be triggered so that t yassword. If the authentication succeeded, then depending on the action e is ‘Authentication’ he user can re-enter | he customer is goin; o take, the state can be either “Checking balance’, “Deposit”, or “Withdraw”. When the use inishes using the ATM, a transition will be triggered and the system goes 0 “Idle” state. Lasth here is a transition from “Idle” state to the final state on the top right, which represents whe: he ATM is shut down. Fig. 3.11 A sequence diagram of an ATM system line, called a lifeline. Messages are drawn as a horizontal arrow from a sender to a receivet Figure 3.11 shows a sequence diagram when a customer uses an ATM to check the balance of his bank account. There are three entities: customer, bank system, and database. Afte: inserting the card into the ATM, the customer enters the PIN. The bank system will check the PIN against the database. If the PIN is correct, the customer will be granted access, then car request his balance. The bank system will get the balance from the database, then inform the customer. If the PIN is wrong, the customer cannot gain access, and has to enter the PIN again To represent the loop (keep entering PIN) and if-then-else logic (correct or wrong PIN), twc kinds of combined fragments (loop and alt) are used. Each is represented as a box. A loof combined fragment executes multiple times. An alt (alternatives) combined fragment shows conditional flow by modelling if-then-else logic. In Figure 3.11, the alt combined fragmen is split into two sections by a dashed line: the first section shows the interaction when wrong PIN is enter, while the second section shows the messages when the PIN is correct and the use: requests the balance. Fig. 3.12 An activity diagram of an ATM system 3.5 Design of the Initial Framework Heinze et al. (2000) also suggests a three step process for specifying agent systems: 3.5 Design of the Initial Framework sased on real life and/or theories, economists can formulate the problem as a PGG, and conduc laboratory experiment of that PGG to collect data. Before creating a simulation, the PGC s investigated to create a conceptual model. The next step is to perform analysis to create al nalysis model of the model to be implemented. Based on the analysis model, the simulatio1 ; designed and implemented. The simulation can be parameterized using laboratory dat: \fterwards the simulation is tested and evaluated. After evaluation, the process is started ove rom analysis if needed. This shows the iterative nature of the development process. Afte unning and experimenting with the simulation, the simulated data can be validated agains ne laboratory data. After the interpretation of data, economists gain understanding of the rea fe problems or theory, and can find answers to the initial questions or evidence to support thi voothesis. Table 4.2 Model Level of Detail of Case Study | - KISS approach Table 4.1 Model Scope of Case Study 1 - KISS approach Fig. 4.1 Use case diagram - KISS approach 4.3 Modelling using KISS Approach inclinationToBeAnnoyed, willingToPunish, punishDecision, expectToBePunished required for the punish() operation. 4.3 Modelling using KISS Approach Fig. 4.6 AnyLogic IDE with the project of the KISS approach Table 4.3 Model Scope of Case Study 2 - KIDS approach Table 4.4 Model Level of Detail of Case Study 2 - KIDS approach Identify use cases Fig. 4.8 Class diagram - KIDS approach Fig. 4.10 AnyLogic implementation of KIDS approach Table 4.5 Transitions in the statechart of Person agent variable (default value of false) is the guard for the self-transition of Punish state and ensures hat this transition is only triggered after a punishing decision has been made. isPunishing s set to false every time when entering the Punish state, and set to true after the punishing lecision has been made in the punish() operation. The third variable is punishLock. During ounishment, both the punishing agent and the punished agent will spend their time in Punish ind BePunished state. For every minute the punishing agent is in Punish state, the punished igent spends 3 minutes in BePunished state. There can be the case that the punished agent will ‘eceive multiple punishments and the punished time will overlap. To avoid this situation, a synchronization mechanism using a lock punishLock has been designed so that there can be mly one punishing agent at a time. The punishLock is an integer variable in Main with the Jefault value of 0. The trigger of the transition from Idle to Punish state requires the lock tc se 0. When the punishing agent enters Punish state, it increases the lock by 1. So when one igent is in Punish state, the other agents cannot move from Idle to Punish state. The lock is Jecreased by 1 when the punishing agent moves from Punish to Idle. In addition, the punished igent also increases the lock by | when it moves from Idle to BePunished and decreases the ock by 1 when it changes back to Idle state. With this lock mechanism, one punishing agent -an punish multiple punished agents, and the lock only resets to 0 when all punished agents ire back to Idle. For example, in Figure 4.11, agent A spends | minute to punish agent B, i.e agent B stays in BePunished state for 3 minutes. After a minute in Punish state, agent A spends nother minute to punish agent C. Agent C has to stay in BePunished for 3 minutes. So afte1 2 minutes, agent A finishes punishing and moves to Idle state, but agent B and C are still in 3ePunished state. At that moment, the lock is equal to 2, i.e. other agents cannot punish. When gent C finishes being punished and changes to Idle state, the lock is back to 0. Fig. 4.11 Example of a punishing situation gent C finishes being punished and changes to Idle state, the lock is back to 0. Simulation 2 runs for 15 meals (5 days with 3 meals a day). The mean cooperativenes: of all agents over time is collected in four different scenarios: Simulation 1 and 2 witt or without punishment. In each scenario, the simulations are replicated 100 times and the average cooperativeness of each round (or meal) is calculated. Figure 4.12 illustrates the mear cooperativeness of four agents in the house over time. In the scenarios where punishment i: allowed, the mean cooperativeness is increasing over time and becomes stable at 0.8. In the scenarios without punishment, the cooperativeness in the shared house decreases over time then remains stable at 0.33. Our hypothesis is confirmed: the trending of the cooperativeness it the simulation is the same as the observation in Fehr and Giachter’s experiment. the simulation is the same as the observation in Fehr and Gichter’s experiment. There are two results to be discussed. Firstly, the initial value of mean cooperativeness i: decided by the composition of different strategies and the punishment setting. The coopera tiveness of cooperator is always 1. The initial mean cooperativeness of reciprocators is 0.5 because their initial expected cooperativeness is uniformly distributed from 0 to 1. The initial mean cooperativeness of maximizers is 0 during no punishment and increases to 0.5 when the punishment is enabled in their outcome calculation. Knowing the initial mean cooperativenes: of the three strategies, the initial value of group cooperativness can be calculated. Secondly the reason for the decreased trend of contribution is that the two maximizers cause the mear cooperativeness of the house to be lower than the expectation of the reciprocator. Therefore the reciprocator keeps lowering the decision variable until its expectation is equal to the mear Fig. 4.13 Overview lifecycle of the ABOOMS framework v2 4.7 Framework Revision and the ABOOMS Framework v2 Table 5.2 Model Level of Detail the «extend» relationship, the extending use case, “Punish”, might be performed as part of the “Play a round” use case but is not mandatory. The “Play a round” use case can execute successfully without invoking “Punish”. There are extension points in the flow to specify where the extending use case is executed. In the use case documentation in Section 5.3.3, the inclusion and extension points are mentioned with use case names in square brackets. The strategy change of Full Cooperators and Full Defectors is illustrated in Figure 5.7. The full cooperators do not change their strategy. The agents that use FD strategy would change to WCC strategy when changing to “Cautious” state, and change back to FD strategy when changing to “Normal” state. WCC’ is the same strategy as WCC. The reason for a different name is that WCC’ is a dummy-state that supports the full defector to switch back to FD without being affected by the strategy change of conditional cooperators. Fig. 5.8 An AnyLogic screenshot of the simulation of case study 2 Fig. 5.10 An example of statechart with probability transitions sensitivity statechart (Figure 5.9c) changes to a new state, it sends a message to stra egy statechart, and a strategy change might be triggered. The message contained a String of either "normal" or "cautious" to inform the new state of punishment sensitivity statec Nart. Although transitions receive this message, there are still conditions, such as level of anxiety and probability, that need to be satisfied before transitions can be triggered. Therefore, t are guard constraints, which will be evaluated before triggering a transition. nere Even though strategy change is implemented using a statechart, there is another approach Fig. 5.12 Average investment over four blocks Because in the second block of this particular experiment, the punishment was implemented at rounds 12 and 14, Figure 5.13 shows that the average investment has a sharp rise after rounds 12 and 14. In those two rounds, it appeared that free riding agents, especially low-anxiety agents, got punished, switched to more cooperative strategies, and contributed more in the next round. If this is a lab experiment, we would not be able to investigate much further. But since we use an agent-based model, we can analyse the decision making process of agents better. Figure 5.14 shows the states of punishment sensitivity statechart of all agents. In the first 10 rounds, all agents were in Normal state. When there was a threat of punishment, high-anxiety agents changed to Cautious state. In rounds 12 and 14, when the punishment was implemented, free riding agents got punished and changed to Cautious state. The more agents were in Cautions state, the more contribution there was overall. The strategy change of! conditional cooperators, who are the majority, plays a crucial role in the contribution in the system. Figure 5.15 shows the strategy change of high and low anxiety agents. High-anxiety agents changed to a more cooperative strategy when there was a threat of punishment then changed the strategy again after being cautious for 10 rounds. Low-anxiety agents still used the same strategy even when there was a threat of punishment. They only changed wher punishment was implemented (rounds 12 and 14): the more agents used SCC, the less agents used WCC. After 3 rounds of being cautious, low-anxiety agents changed back to Normal state and used less cooperative strategies. 5.6.2 Experiments with different punishment conditions results to understand more about free riding behaviours in various punishment conditions. Fig. 5.17 Implemented punishment blocks occurred periodically with randomly two out of ten punishment games per block Fig. 5.18 Implemented punishment blocks occurred periodically with randomly five out of ten punishment games per block Table 5.6 Experiments with different ratios of anxiety Fig. 5.19 Total investment in different ratios of strategy changing the ratios of high and low anxiety of conditional cooperators (CSS, NCC, WCC). In the second experiment, 80% of conditional cooperators were high anxiety and 20% were low anxiety. In the third experiment, 20% of conditional cooperators were high anxiety and 80% were low anxiety. Table 5.6 shows the three experiments and corresponding percentages of anxiety levels. Fig. 5.21 Overview lifecycle of the ABOOMS framework v3 5.8 Framework Revision and the ABOOMS Framework v3 The experiment of Oprea et al. reported that continuous time slightly improves contribution compared to discrete time but cannot reach mutual cooperation, and free-form communication substantially raises the contribution comparing to no-communication (Figure 6.1). Since the ‘ree-form communication contains too much noise, it would be very hard to identify the slements that drive contribution in this setting. Therefore, this case study focuses on the 10-communication setting. The experiment shows that in no-communication experiments, the oarticipants start to use pulses in contribution as an effort to communicate. A pulse is a situation vhen one increases his contribution to a very high value (often the maximum value) then drops Jown to the previous level immediately. These pulses may be used as signals of cooperation, ut we can not be certain that all the pulses are seen or how they are interpreted by other parties. [he participants could miss the pulse, did not understand the signal, or saw it but still waited for responses from others. Without communication, the intention of the pulses is ambiguous. Fig. 6.1 (a) Median and (b) mean contributions by period and treatment (Oprea et al., 2014) for responses from others. Without communication, the intention of the pulses is ambiguous. Fig. 6.2 Screenshot of a continuous-time laboratory experiment (Oprea et al., 2014) Fig. 6.3 Median and mean of contributions over 10 sub-periods of 1 minute (Oprea et al., 2014) Because there are major differences in behaviour between types of Person agents, such as pulse behaviour of GCCs compared to dissatisfied behaviour of cooperators, inheritance is employed in the structural design to capture the semantics of the type hierarchy. Person is the superclass, and each of four types is a subclass (i.e. aggregation relationship between Person and Group class). So there are seven classes in total, shown in the class diagram Figure 6.5 The four subclasses inherit attributes and behaviours of Person superclass, but still have their own attributes and behaviours. For example, the group variable of Person class is inherited by subclasses, but contribute() operation is overriden by each subclass to implement their own haha Are subclasses, but contribute() operation is overriden by each subclass to implement their own 6.4.2 Behavioural design Table 6.5 Parameters of Experiment 1, 2 ,and 3 To explain the effect of GCCs, an analysis of the contribution dynamics within groups is 1eeded. Figure 6.9 presents contribution dynamics within six groups that are chosen from he simulation runs. Groups 1 and 2 have no GCCs. Group | has more cooperators so the -ontribution is high; Group 2 has more defectors so the contribution is low. When GCCs are ntroduced into the game, we have Groups 3, 4, 5, and 6. The first three experiments were set up with the parameters in Table 6.5. For each experiment, there were 10 replications and the contribution was collected every second. Then the average contributions over time of 10 replications were calculated and shown in Figure 6.8. The first experiment has the ratio of strategies as suggested in the conceptual modelling, and the percentages are all set to 50% for a balance mix of different rates. In order to compare to the situation when GCCs are absent in the population, in the second experiment, 400 GCCs are changed to CCs. Thus there are 700 CCs in total. In Experiment 2, the contribution decreases and reach an equlibrium. The presence of GCCs in Experiment | has lead to a higher value of contribution. The contribution slightly decrease then stabilizes. This leads to our first result: Result 1: The presence of GCCs increases group contribution. contribution. The contribution slightly decrease then stabilizes. This leads to our first result: Table 6.6 Parameters of Experiment 4 After experimenting with the evaluation rate of CCs 1n the third experiment, the fourth ex- periment is introduced to investigate the evaluation rate and generosity rate of GCCs. The parameters set-up for the fourth experiment are shown in Table 6.6. For each combination of the two percentages, the simulation was run for 10 replications, and the average contribution of each replication was collected. Then the average contributions of all 10 replications was averaged to create Table 6.7. The contribution is increasing as both percentages increase from 0% to 100%. It is highest when both percentages are 100%, i.e. all GCCs have slow evaluation rate (evaluateRate=5) and stay generous for a longer period (stayGenerousRate=5). Result 3: For GCCs, slow evaluation and staying generous longer leads to higher group contribution. “or GCCs, slow evaluation and staying generous longer leads to higher group contribution. percentageLongerStayGenerous Table 6.8 Parameters of Experiment 5 means less time staying in Generous state). This is consistent with Result 3. The y axis is the contribution ranged from 0 to 25. This appendix presents the figures of contributions in non-communication continuous-time PGG (Oprea et al., 2014). There are five groups corresponding to five figures. In each figure, there are four colour-coded lines representing four players. The x axis is the time in seconds. The y axis is the contribution ranged from 0 to 25.
