Self-organising coordination

We design a self-exploratory reinforcement learning (RL) framework, based on the Q-learning algorithm, that enables the base station (BS) to choose a suitable modulation and coding scheme (MCS) that maximizes the spectral efficiency while maintaining a low block error rate (BLER). In this framework, the BS chooses the MCS based on the channel quality indicator (CQI) reported by the user equipment (UE). A transmission is made with the chosen MCS and the results of this transmission are converted by the BS into rewards that the BS uses to learn the suitable mapping from CQI to MCS. Comparing with a conventional fixed look-up table and the outer loop link adaptation, the proposed framework achieves superior performance in terms of spectral efficiency and BLER.

We design a self-exploratory reinforcement learning (RL) framework, based on the Q-learning algorithm, that enables the base station (BS) to choose a suitable modulation and coding scheme (MCS) that maximizes the spectral efficiency while maintaining a low block error rate (BLER). In this framework, the BS chooses the MCS based on the channel quality indicator (CQI) reported by the user equipment (UE). A transmission is made with the chosen MCS and the results of this transmission are converted by the BS into rewards that the BS uses to learn the suitable mapping from CQI to MCS. Comparing with a conventional fixed look-up table and the outer loop link adaptation, the proposed framework achieves superior performance in terms of spectral efficiency and BLER.

We consider a distributed SON (D-SON) architecture where the interaction of different self-organizing network (SON) functions negatively affect the performances of the system. This is referred to in 3rd Generation Partnership Project (3GPP) as a SON conflict, which needs to be handled by means of a self-coordination framework. We focus on a functional architecture and a theoretical framework based on the theory of Markov decision process (MDP) for the self-coordination of different actions taken by different SON functions. In order to cope with the complexity of the overall SON problem, we subdivide the global MDP modeling the long-term evolution (LTE)-enhanced node base station (eNB) onto simpler subMDPs modeling the different SON functions. Each sub-problem is defined as a subMDP and solved independently by means of reinforcement learning (RL), and their individual policies are combined to obtain a global policy. This combined policy can execute several actions per state but can introduce policy conflicts. We focus on the specific SON conflict generated by the concurrent execution of coverage and capacity optimization (CCO) and inter-cell interference coordination (ICIC) SON functions, which may require to update the same parameter, i.e., the transmission power level. The coordination among the different actions selected by the conflicting use cases is achieved by means of a coordination game where the players are the subMDPs and the actions and rewards are those provided by means of a RL approach. Performance evaluation is carried out in a ns3 release 8 compliant LTE system simulator, and it shows that our self-coordination approach provides satisfying solutions in terms of system performances for both the conflicting SON functions.

which we would like to understand, capture, then bring to computational systems Nature-Inspired Computing (NIC) For instance, NIC [Liu and Tsui, 2006] summarises decades of research activities putting emphasis on autonomy of components, and on self-organisation of systems Omicini (DISI, Univ. Bologna) 17-Nature-inspired Coordination A.Y. 2012/2013 4 / 46 Why? Why Coordination Models? Interaction most of the complexity of complex computational systems comes from interaction [Omicini et al., 2006] along with an essential part of their expressive power [Wegner, 1997] Coordination since coordination is essentially the science of managing the space of interaction [Wegner, 1997] coordination models and languages [Ciancarini, 1996] provide abstractions and technologies for the engineering of complex computational systems [Ciancarini et al., 2000] Omicini (DISI, Univ. Bologna) 17-Nature-inspired Coordination A.Y. 2012/2013 5 / 46 Why? Why Nature-inspired Coordination? Coordination issues in natural systems coordination issues did not first emerge in computational systems [Grassé, 1959] noted that in termite societies "The coordination of tasks and the regulation of constructions are not directly dependent from the workers, but from constructions themselves." Coordination as the key issue many well-known examples of natural systems-and, more generally, of complex systems-seemingly rely on simple yet powerful coordination mechanisms for their key features-such as self-organisation it makes sense to focus on nature-inspired coordination models as the core of complex nature-inspired computational systems Omicini (DISI, Univ. Bologna) 17-Nature-inspired Coordination A.Y

To face the challenges of knowledge-intensive environments, we investigate a novel self-organising knowledge-oriented (SOKO) model, called Molecules of Knowledge (MoK for short). In MoK, knowledge atoms are generated by knowledge sources in shared spaces-compartments-, self-aggregate to shape knowledge molecules, and autonomously move toward knowledge consumers.

Coordination of several agents in accessing a limited resource is a common problem among various systems, such as mass transportation, communication networks, and stock markets. In the absence of a central coordinator, the primary challenge of the problem is to bring equilibrium among agents in accessing a limited shared resource. An imbalanced and unfair access policy can cause congestion, starvation, and performance degradation. TheEl Farol Bar Problem is the generic description of this problem, in which a group of citizens may attend a bar with a limited capacity on a specific day of week. If the number of attendees is equal or under the capacity, they will have a pleasant time, and their individual utilities will increase. Consequently, the social utility will raise. Otherwise, the individual utility of the participants will decrease, and the social utility declines as well. Four main evaluation metrics are introduced in the literature, which are social utility, individual utility, starvation avoidance, and the convergence speed. Most of the studies in this topic consider the social utility metric as the primary optimization objective. Some others apply the lower limit restriction for an agent's access request. In this paper, we devised a new two-dimensional approach called Social Coordination (SoCo). In the first dimension, we define a new function, called Effect(), that plays a determinative role in choosing the strategy for the current action. In the second dimension, we define a new social coordination constraint that boosts the system to achieve the entire equilibrium, in which near optimum status in social and individual utilities are reached without any starvation cases. The new approach not only attempts to improve the social utility, but also considers the individual utility and starvation as the optimization goals. The simulation results show that SoCo improves the social utility by 57.61% compared to the similar approaches. We also perform a range of simulations to analyze the behavior of our approach in different history sizes and agent populations. The simulations demonstrate that the maximum starvation length of agents in SoCo is 7.93 times less than the similar methods.

Based on two intuitions -(i) event-driven systems and multi-agent systems are two computational paradigms that are amenable of a coherent interpretation within a unique conceptual framework; (ii) integrating the two simulation approaches can lead to a more expressive and powerful simulation framework -we propose a computational model integrating Discrete-Event Simulation (DES) and Multi-Agent Based Simulation (MABS), based on an extension of the Gillespie's stochastic simulation algorithm.

Abstract An interesting application of self-organization techniques is in the context of coordination languages and models, which aims at developing tools (languages, models, infrastructures) to flexibly manage the interaction of components in distributed systems. In a coordinated system, the environment is filled with coordination media--eg tuple spaces or interaction channels--enacting coordination laws that are typically reactive, deterministic, and global.

To face the challenges of knowledge-intensive environments, we investigate a novel self-organising knowledge-oriented (SOKO) model, called Molecules of Knowledge (MoK for short). In MoK, knowledge atoms are generated by knowledge sources in shared spaces — compartments —, self-aggregate to shape knowledge molecules, and autonomously move toward knowledge consumers.