Cooperative Multiagent Systems

This paper treats the coordination of Emergency Medical Assistance (EMA) and hospitals for after-hours surgeries of urgent patients arriving by ambulance. A standard hospital approach during night-shifts is to have standby surgery teams come to hospital after alert to cover urgent cases that cannot be covered by the in-house surgery teams. This approach results in a considerable decrease in staffing costs in respect to having sufficient permanent in-house staff. Therefore, coordinating EMA and the hospitals in a region with their outhouse staff with the objective to have as fast urgent surgery treatments as possible with minimized cost is a crucial parameter of the medical system efficiency and as such deserves a thorough investigation. In practice, the process is manual and the process management is case-specific, with great load on human phone communication. In this paper, we propose a decision support system for the automated coordination of hospitals, surgery teams on standby from home, and ambulances to decrease the time to surgery of urgent patients. The efficiency of the proposed model is proven over simulation experiments.

Loosely-interconnected cooperative systems such as cable robots are particularly susceptible to uncertainty. Such uncertainty is exacerbated by addition of the base mobility to realize reconfigurability within the system. However , it also sets the ground for predictive base reconfigu-ration in order to reduce the uncertainty level in system response. To this end, in this paper we systematically quantify the output wrench uncertainty based on which a base re-configuration scheme is proposed to reduce the uncertainty level for a given task (uncertainty manipulation). Variations in tension and orientation of the cables are considered as the primary sources of the uncertainty responsible for non-deterministic wrench output on the platform. For non-optimal designs/configurations, this may require complex control structures or lead to system instability. The force vector corresponding to each agent (e.g., pulley and cable) is modeled as random vector whose magnitude and orientation are modeled as random variables with Gaussian and von Mises distributions, respectively. In a probabilistic framework , we develop the closed-form expressions of the mean and variances of the output force and moment given the current state (tension and orientation of the cables) of the system. This is intended to enable the designer to efficiently characterize an optimal configuration (location) of the bases in order to reduce the overall wrench fluctuations for a specific task. Numerical simulations as well as real experiments with multiple iRobots are performed to demonstrate the effectiveness of the proposed approach.

We apply diffusion strategies to propose a cooperative reinforcement learning algorithm, in which agents in a network communicate with their neighbors to improve predictions about their environment. The algorithm is suitable to learn off-policy even in large state spaces. We provide a mean-square-error performance analysis under constant step-sizes. The gain of cooperation in the form of more stability and less bias and variance in the prediction error, is illustrated in the context of a classical model. We show that the improvement in performance is especially significant when the behavior policy of the agents is different from the target policy under evaluation.

The exponential growth in mobile communications networks and recent demand for mobile services has led to an increase in the demand for spectrum. This demand is expected to increase during the deployment of the proposed 5G technology. However, the spectrum allocation policies that are currently being deployed by mobile network operators (MNOs) are on exclusive basis. In this project, we will focus on the techniques proposed to efficiently deploy and implement shared spectrum between cellular network operators (spectrum pooling and mutual renting), evaluate state of the art and propose a more efficient scheme to optimize the usage of the shared spectrum in mutual renting between MNOs.

Mobile robots are increasingly being used for tasks like remote surveillance, sensing and maintenance. Some of these tasks are critical and require intelligent decision making for successful completion. It is not always possible to rely exclusively on robot level intelligence to make high impact decisions and hence human supervision is needed during task execution. To facilitate human-in-the-loop task servicing, the task executing robot is required to remain connected to a remotely located human operator.

One important class of problems in Multi-Agent Systems (MASs) is planning, that is constructing an optimal policy for each agent with the objective of reaching some terminal goal state. The key aspect of multi-agent planning is coordinating the actions of the individual agents. This coordination may be done through communication, learning, or conventions imposed at design time. In this paper we present a new taxonomy of MASs that is based on the notions of optimality and rationality. A framework that describes the interactions between the agents and their environment is given, along with a reinforcement learning-based algorithm (Q-learning) for learning a joint optimal plan. Finally, we give some experimental results on grid games that show the convergence of this algorithm.

This paper treats the coordination of Emergency Medical Assistance (EMA) and hospitals for after-hours surgeries of urgent patients arriving by ambulance. A standard hospital approach during night-shifts is to have standby surgery teams come to hospital after alert to cover urgent cases that cannot be covered by the in-house surgery teams. This approach results in a considerable decrease in staffing costs in respect to having sufficient permanent in-house staff. Therefore, coordinating EMA and the hospitals in a region with their outhouse staff with the objective to have as fast urgent surgery treatments as possible with minimized cost is a crucial parameter of the medical system efficiency and as such deserves a thorough investigation. In practice, the process is manual and the process management is case-specific, with great load on human phone communication. In this paper, we propose a decision support system for the automated coordination of hospitals, surgery teams on standby from home, and ambulances to decrease the time to surgery of urgent patients. The efficiency of the proposed model is proven over simulation experiments.

We apply diffusion strategies to propose a cooperative reinforcement learning algorithm, in which agents in a network communicate with their neighbors to improve predictions about their environment. The algorithm is suitable to learn off-policy even in large state spaces. We provide a mean-square-error performance analysis under constant step-sizes. The gain of cooperation in the form of more stability and less bias and variance in the prediction error, is illustrated in the context of a classical model. We show that the improvement in performance is especially significant when the behavior policy of the agents is different from the target policy under evaluation.