Wake-Up Scheduling

Reinforcement Learning (RL) is a promising technology that has made a significant improvement in Industrial Internet of Things (IIoT) based applications by providing a high level of accuracy and optimal resource utilization while handling complicated problems efficiently. This paper aims to help future researchers to design efficient communication protocols for IIoTbased networks by exploiting the power of the RL technology. We provide a critical mini review on the recent RL-based MAC and routing protocols intended for IIoT-based applications, focus on Wireless Sensor Networks (WSN), and show through in-depth analysis how RL is exploited to address their related issues. The comparison between the reviewed works shows that the learning model's complexity, real-time communications, and security should deserve more consideration during the development of communication protocols to meet the specific needs of IIoT applications.

A wireless sensor network is a collection of small sensor nodes that have limited energy and are usually not rechargeable. Because of this, the lifetime of wireless sensor networks has always been a challenging area. One of the basic problems of the network has been the ability of the nodes to effectively schedule the sleep and wake-up time to overcome this problem. The motivation behind node sleep or wake-up time scheduling is to take care of nodes in sleep mode for as long as possible (without losing data packet transfer efficiency) and thus extend their useful life. This research going to propose scheduling of nodes sleeps and wake-up time through reinforcement learning. This research is not based on the nodes' duty cycle strategy (which creates a compromise between data packet delivery and nodes energy saving delay) like other existing researches. It is based on the research of reinforcement learning which gives independence to each node to choose its own activity from the transmission of packets, tuning or sleep node in each time band which works in a decentralized way. The simulation results show the qualified performance of the proposed algorithm under different conditions.

A wireless sensor network is a collection of small sensor nodes that have limited energy and are usually not rechargeable. Because of this, the lifetime of wireless sensor networks has always been a challenging area. One of the basic problems of the network has been the ability of the nodes to effectively schedule the sleep and wake-up time to overcome this problem. The motivation behind node sleep or wake-up time scheduling is to take care of nodes in sleep mode for as long as possible (without losing data packet transfer efficiency) and thus extend their useful life. This research going to propose scheduling of nodes sleeps and wake-up time through reinforcement learning. This research is not based on the nodes' duty cycle strategy (which creates a compromise between data packet delivery and nodes energy saving delay) like other existing researches. It is based on the research of reinforcement learning which gives independence to each node to choose its own activity from the t...

Wireless sensor networking is a viable communication technology among low-cost and energy-limited sensor nodes deployed in an environment. Due to high operational features, the application area of this technology is extended significantly but with some energy related challenges. One main cause of the nodes energy wasting in these networks is idle listening characterized with no communication activity. This drawback can be mitigated by the means of energy-efficient multiple access control schemes so as to minimize idle listening. In this paper, we discuss the applicability of distributed learning algorithms namely reinforcement learning towards multiple access control (MAC) in wireless sensor networks. We perform a comparative review of relevant work in the literature and then present a cooperative multi agent reinforcement learning framework for MAC design in wireless sensor networks. Accordingly, the paper concludes with some major challenges and open issues of distributed MAC design using reinforcement learning.

We propose a novel approach to design a coloring Wakeup Schedule (CWS) for the radio of the nodes in wireless sensor networks, in order to reduce the end-to-end latency with energy efficient data transmission. In this letter, a novel Coloring Wake-up Schedule (CWS) is presented for the radio of the nodes in the wireless sensor networks. The aim of this approach is reducing the end-to-end latency and energy consumption in the data transmission. In the CWS, a particular color is assigned to each cell of the nodes. By this, the packet of the node can transfer along the shortest path to the sink node; the nodes find their neighbour according to their transmission range and choose the neighbour that is near than any nodes to the sink node. Therefore the mentioned-schedule decreases the useless activities of the nodes which caused to save the energy. Also, a specific grid is defined for the situation of the CWS nodes to improve the routing. The various simulations prove the usefulness of ...

The inherent many-to-one flow of traffic in Wireless Sensor Networks produces a skewed distribution of energy consumption rates leading to the early demise of those sensors that are critical to the ability of surviving nodes to communicate their readings to the data collection center. Numerous previous approaches aimed at balancing the consumption of energy in wireless networks are based on a linear programming-based minmax formulation that seeks to maximize the minimum lifetime of sensors in a network. However, this approach fails to provide a clear picture of the cost at which this optimization is achieved and focuses attention on a single sensor, the minimum lifetime sensor. This paper makes two contributions; 1) it puts forward a new understanding of sensor network lifetime based on statistical measures, mean and variance, of node power consumption rates that provides a more inclusive view of the consumption rates, and 2) it provides an optimal quadratic programming (QP) formulation that provides an upper bound on the lifetime under a given set of topological and energy budgetary constraints. Our results demonstrate that the QP formulation has the ability to provide a soft trade-off between the mean and variance of power consumption rates.

Decentralised, energy-efficient synchronisation of duty-cycles is a difficult task in Wireless Sensor Networks (WSNs). Popular protocols for this task are inspired by the synchronisation mechanism of fireflies. One such protocol-the emergent broadcast slot (EBS) scheme-achieves high synchronisation levels at low energy cost. However, its performance heavily depends on its configuration. To understand the impact of the protocol parameters better, we analyse a MAC-level implementation of EBS using Castalia, a low-level WSN simulation framework. Moreover, we show work in progress on a scalable GSMP model of EBS.

This letter presents a simple model for determining energy efficient random sleep-awake schedules. Random sleepawake schedules are more appropriate for sensor networks, where the time of occurrence of an event being monitored, e.g., the detection of an intruder, is unknown a priori, and the coordination among nodes is costly. We model the random sleepawake schedule as a two state Markov process, and maximize the probability of the transmission of sensed data by a given deadline. Our results indicate that for a given duty cycle, the optimal policy is to have infrequent transitions between sleep and awake modes, if the average number of packets sent is greater than the mean number of slots the node is awake.

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor's sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.

Synchronicity is one of the essential basic services to support the main duties of Wireless Sensor Networks (WSNs). Synchronicity is the ability to arrange simultaneously collective actions in WSNs. A high-rate data sampling to analyze the seismic structure and volcanic monitoring is the important applications requiring a synchronicity. However, most of the existing synchronicity algorithm is still executed in a flat network, so that it requires a long time to achieve a synchronous condition. To increase the convergence rate, the synchronicity can be executed concurrently through a clustering scheme approach. In this work, the such scheme is called as the Node Clustering based on Initial Phase Proximity for Reachback Firefly Synchronicity (NCIPP-RFS). The NCIPP-RFS solves in three steps: (1) constructing the node clustering, (2) intra-cluster synchronicity, and (3) inter-cluster synchronicity. The NCIPP-RFS method is based on the firefly-inspired algorithm. The fireflies are a speci...

In this paper authors have elaborated on objective of MAC protocol, energy efficiency of MAC protocol and comparison of MAC layer protocols such as S-MAC and D-MAC for WSNs. The primary goal of the S-MAC design is to improve energy efficiency while maintaining good scalability and collision avoidance. To achieve this goal, S-MAC tries to reduce energy consumption from all the major sources that causes inefficient use o f energy. D-MAC protocol was proposed to address the data forwarding interruption problem in multihop data delivery and its primary goal is to achieve both energy efficiency and low latency. Authors have tried to explain how D-MAC is more energy efficient than S-MAC and also elaborated on how performance of D-MAC can be improved using single or multiple mobile element based data gathering Technique.

In this paper authors have elaborated on objective of MAC protocol, energy efficiency of MAC protocol and comparison of MAC layer protocols such as S-MAC and D-MAC for WSNs. The primary goal of the S-MAC design is to improve energy efficiency while maintaining good scalability and collision avoidance. To achieve this goal, S-MAC tries to reduce energy consumption from all the major sources that causes inefficient use o f energy. D-MAC protocol was proposed to address the data forwarding interruption problem in multihop data delivery and its primary goal is to achieve both energy efficiency and low latency. Authors have tried to explain how D-MAC is more energy efficient than S-MAC and also elaborated on how performance of D-MAC can be improved using single or multiple mobile element based data gathering Technique.

Wireless sensor networking is a viable communication technology among low-cost and energy-limited sensor nodes deployed in an environment. Due to high operational features, the application area of this technology is extended significantly but with some energy related challenges. One main cause of the nodes energy wasting in these networks is idle listening characterized with no communication activity. This drawback can be mitigated by the means of energy-efficient multiple access control schemes so as to minimize idle listening. In this paper, we discuss the applicability of distributed learning algorithms namely reinforcement learning towards multiple access control (MAC) in wireless sensor networks. We perform a comparative review of relevant work in the literature and then present a cooperative multi agent reinforcement learning framework for MAC design in wireless sensor networks. Accordingly, the paper concludes with some major challenges and open issues of distributed MAC design using reinforcement learning.

Biologically inspired self-organization methods can help to manage the access control to the shared communication medium of Wireless Sensor Networks. One lightweight approach is the primitive of desynchronization, which relies on the periodic transmission of short control messages – similar to the periodical pulses of oscillators. is primitive of desynchronization has already been successfully implemented as MAC protocol for single-hop topologies. Moreover, there are also some concepts of such a protocol for multi-hop topologies available. However, the existing implementations may handle just a certain class of multi-hop topologies or are not robust against topology dynamics. In addition to the sophisticated access control of the sensor nodes of a Wireless Sensor Network in arbitrary multi-hop topologies, the communication protocol has to be lightweight, applicable, and scalable. ese characteristics are of particular interest for distributed and randomly deployed networks (e.g., by ...

In their EWSN'07 paper [1], Giusti et al. proposed a decentralized wake-up scattering algorithm for temporally spreading the intervals in which the nodes of a wireless sensor network (WSN) are active, and showed that the resulting schedules significantly improve over the commonly-used random ones, e.g., by providing greater area coverage at less energy costs. However, an open question remained about whether further improvements are possible. Here, we complete the work in [1] by providing a (centralized) optimal solution that constitutes a theoretical upper bound for wake-up scattering protocols. Simulation results shows that the decentralized algorithm proposed in [1] comes within 4% to 11% of the optimum. Moreover, we show that the modeling framework we use to derive the solution, based on integer programming techniques, allows for a particularly efficient solution. The latter result discloses important opportunities for the practical utilization of the model. The model is also general enough to encompass alternative formulations of the problem.

In their EWSN'07 paper [1], Giusti et al. proposed a decentralized wake-up scattering algorithm for temporally spreading the intervals in which the nodes of a wireless sensor network (WSN) are active, and showed that the resulting schedules significantly improve over the commonly-used random ones, e.g., by providing greater area coverage at less energy costs. However, an open question remained about whether further improvements are possible. Here, we complete the work in [1] by providing a (centralized) optimal solution that constitutes a theoretical upper bound for wake-up scattering protocols. Simulation results shows that the decentralized algorithm proposed in [1] comes within 4% to 11% of the optimum. Moreover, we show that the modeling framework we use to derive the solution, based on integer programming techniques, allows for a particularly efficient solution. The latter result discloses important opportunities for the practical utilization of the model. The model is also general enough to encompass alternative formulations of the problem.

In this paper authors have elaborated on objective of MAC protocol, energy efficiency of MAC protocol and comparison of MAC layer protocols such as S-MAC and D-MAC for WSNs. The primary goal of the S-MAC design is to improve energy efficiency while maintaining good scalability and collision avoidance. To achieve this goal, S-MAC tries to reduce energy consumption from all the major sources that causes inefficient use o f energy. D-MAC protocol was proposed to address the data forwarding interruption problem in multihop data delivery and its primary goal is to achieve both energy efficiency and low latency. Authors have tried to explain how D-MAC is more energy efficient than S-MAC and also elaborated on how performance of D-MAC can be improved using single or multiple mobile element based data gathering Technique.

In this paper authors have elaborated on objective of MAC protocol, energy efficiency of MAC protocol and comparison of MAC layer protocols such as S-MAC and D-MAC for WSNs. The primary goal of the S-MAC design is to improve energy efficiency while maintaining good scalability and collision avoidance. To achieve this goal, S-MAC tries to reduce energy consumption from all the major sources that causes inefficient use o f energy. D-MAC protocol was proposed to address the data forwarding interruption problem in multihop data delivery and its primary goal is to achieve both energy efficiency and low latency. Authors have tried to explain how D-MAC is more energy efficient than S-MAC and also elaborated on how performance of D-MAC can be improved using single or multiple mobile element based data gathering Technique.

With the advent of the Internet of Things (IoT), an increasing number of energy harvesting methods are being used to supplement or supplant battery based sensors. Energy harvesting sensors need to be configured according to the application, hardware, and environmental conditions to maximize their usefulness. As of today, the configuration of sensors is either manual or heuristics based, requiring valuable domain expertise. Reinforcement learning (RL) is a promising approach to automate configuration and efficiently scale IoT deployments, but it is not yet adopted in practice. We propose solutions to bridge this gap: reduce the training phase of RL so that nodes are operational within a short time after deployment and reduce the computational requirements to scale to large deployments. We focus on configuration of the sampling rate of indoor solar panel based energy harvesting sensors. We created a simulator based on 3 months of data collected from 5 sensor nodes subject to different lighting conditions. Our simulation results show that RL can effectively learn energy availability patterns and configure the sampling rate of the sensor nodes to maximize the sensing data while ensuring that energy storage is not depleted. The nodes can be operational within the first day by using our methods.We show that it is possible to reduce the number of RL policies by using a single policy for nodes that share similar lighting conditions. CCS CONCEPTS • Computing methodologies → Reinforcement learning; • Computer systems organization → Sensor networks;

With the advent of the Internet of Things (IoT), an increasing number of energy harvesting methods are being used to supplement or supplant battery based sensors. Energy harvesting sensors need to be configured according to the application, hardware, and environmental conditions to maximize their usefulness. As of today, the configuration of sensors is either manual or heuristics based, requiring valuable domain expertise. Reinforcement learning (RL) is a promising approach to automate configuration and efficiently scale IoT deployments, but it is not yet adopted in practice. We propose solutions to bridge this gap: reduce the training phase of RL so that nodes are operational within a short time after deployment and reduce the computational requirements to scale to large deployments. We focus on configuration of the sampling rate of indoor solar panel based energy harvesting sensors. We created a simulator based on 3 months of data collected from 5 sensor nodes subject to different lighting conditions. Our simulation results show that RL can effectively learn energy availability patterns and configure the sampling rate of the sensor nodes to maximize the sensing data while ensuring that energy storage is not depleted. The nodes can be operational within the first day by using our methods.We show that it is possible to reduce the number of RL policies by using a single policy for nodes that share similar lighting conditions. CCS CONCEPTS • Computing methodologies → Reinforcement learning; • Computer systems organization → Sensor networks;

Biologically inspired self-organization methods can help to manage the access control to the shared communication medium of wireless ad-hoc networks. One lightweight method is the primitive of desynchronization, which has already been implemented as MAC protocol for single-hop topologies successfully. Here, each periodically transmitting node is able to establish a collision-free TDMA schedule autonomously. However, multi-hop topologies are more realistic, but also more difficult to handle. For instance, the hidden terminal problem is inherent in such topologies and complicates an implementation of this primitive as MAC protocol for multi-hop topologies: Each node requires knowledge about its two-hop neighborhood to establish a collision-free TDMA schedule. Moreover, the problem of stale information is inherent in the primitive of desynchronization and even could destabilize the whole system. In this paper we describe our experience when extending a single-hop MAC protocol based on ...

Biologically inspired self-organization methods can help to manage the access control to the shared communication medium of wireless ad-hoc networks. One lightweight method is the primitive of desynchronization, which has already been implemented as MAC protocol for single-hop topologies successfully: periodically transmitting nodes establish a collision-free TDMA schedule autonomously. However, multi-hop topologies are more realistic, but each node now requires knowledge about its two-hop neighborhood to solve the hidden terminal problem. In this paper we describe our experience using an extended version of a MAC protocol for multi-hop topologies based on the primitive of desynchronization. We identified stale information as pitfall of desynchronization when using phase shift propagation to solve the hidden terminal problem in multi-hop topologies. As a result, we included a refractory threshold to manage stale information at the self-organized MAC protocol based on the primitive o...

One main characteristic of aWireless Sensor Network (WSN) is the wireless communication of the participating sensor nodes. Especially for complex and multi-hop topologies, a MAC protocol is essential to provide a (more or less) collision-free access to the shared communication medium. We prefer a periodical and self-organized MAC protocol because of its robustness, adaptivity and scalability. Another important service for various WSN applications is time synchronization. For this reason, several synchronization protocols and techniques have been investigated already. In this paper we present our real-world experiences when integrating a simple time-stamped synchronization protocol into a self-organized and periodical MAC protocol. This integrative approach requires just marginal additional data, but achieves quite accurate synchronization results. The problems and experiences during our real-world implementation are discussed in detail within this paper.

One of the most challenging research tasks in the field of wireless sensor networks is controlling the power consumption of batteries and prolonging network lifetime. For sensor networks that consist of a large number of sensor nodes, research on bio-inspired self-organization methods has attracted attention due to the potential applicability of such methods. In this paper, we focus on the calling behavior of Japanese tree frogs. They are known to make calls alternately with their neighbors in order to raise the probability of mating. This behavior can be applied to phase control that realizes collision free transmission scheduling in wireless communication. These frogs also display a type of behavior known as satellite behavior, where a frog stops calling once it detects the calls of other neighboring frogs. This behavior can be applied in the design of an energy-efficient sleep control mechanism that provides adaptive operation periods. We propose a self-organizing scheduling scheme inspired by Japanese tree frog calling behavior for energy-efficient data transmission in wireless sensor networks. Simulation results show that our proposed sleep control method prolongs network lifetime by a factor of 6.7 as compared with the method without sleep control for a coverage ratio of 80%.

Current approaches in reinforcement learning that combine MDP reward with human reinforcements assume both signals to be complementary. They rely on the fact that the system and the human have the same desire of how a particular task should be completed. We present the case where these reward signals are conflicting, i.e. the goals of the agent are not aligned with the interests of the human trainer. More precisely, we describe an approach how such problems can be solved by multi-objective reinforcement learning.

Any properly designed network coding technique can result in increased throughput and reliability of multi-hop wireless networks by taking advantage of the broadcast nature of wireless medium. In many inter-flow network coding schemes nodes are encouraged to overhear neighbour's traffic in order to improve coding opportunities at the transmitter nodes. A study of these schemes reveal that some of the overheard packets are not useful for coding operation and thus this forced overhearing increases energy consumption dramatically. In this paper, we formulate network coding aware sleep/wakeup scheduling as a semi Markov decision process (SMDP) that leads to an optimal node operation. In the proposed solution for SMDP, the network nodes learn when to switch off their transceiver in order to conserve energy and when to stay awake to overhear some useful packets. One of the main challenges here is the delay in obtaining reward signals by nodes. We employ a modified reinforcement learning method based on continuous-time Q-learning to overcome this challenge in the learning process. Our simulation results confirm the optimality of the new methodology.

Wireless sensor networks are provided with a limited source of power. The lifetime of such networks is an overwhelming matter in most network applications. This lifetime depends strongly on how efficiently such energy is distributed over the nodes especially during transmitting and receiving data. Each node may route messages to destination nodes either through short hops or long hops. Optimizing the length of these hops may save energy, and therefore extend the lifetime of WSNs. In this paper, we propose a theorem to optimize the hop's length so to make WSN power consumption minimal. The theorem establishes a simple condition on hop's length range. Computer simulation when performing such condition on Mica2 sensors and Mica2dot sensors reveals good performance regarding WSNs energy consumption.

Ad-hoc networks are very dynamic and nodes are entering and exiting network frequently. In such networks self-organization without centralized control is crucial for efficient network operation. To accomplish selforganization in small ad-hoc networks, the pulse-coupled oscillator model based on biological mutual synchronization such as that observed in flashing fireflies, is used. Fireflies are known to emit flashes at regular intervals when isolated, but in the group they entrain the pulsing of their lights to converge upon the same rhythm until synchronicity is reached. This paper investigates the influence of different overlay network topologies to synchronization time and network traffic.

In Multi-Agent Systems (MASs) it is of vital importance that basic network operations are performed without the interference of a central entity (i.e. agent). In this paper we will present how to use a selforganization approach to achieve time synchronization of agents in MASs using a Pulse-Coupled Oscillators (PCO) model that is based on flashing fireflies. Fireflies are known to emit flashes at regular intervals when isolated, but when they are within a group, they converge upon the same rhythm until time synchronization is achieved. This paper investigates how the choice of overlay network topology and metric affects the time synchronization process of agents in MASs.

— Time synchronized nodes in WSN play an important role in many services such as event monitoring, data collection. Because of unsynchronized clock of sensor motes, energy use of network increases considerably. Recently, a scheme for time synchronization using pairwise synchronization was proposed. The scheme used overheard nodes to synchronize nodes in the network thereby relaxing the burden of the network. But as the size of the network increases, nodes cannot perform overhearing from the parent node because of the constraint of limited range of motes. This paper proposes the good at producing time synchronized network using optimally powered overheard nodes in multilayered scenario. These nodes offer greater range to nodes, hence reducing the limitations such as high message delays, low throughput. The control to get an effect of the offered design has been made certain through many simulations. Results make clear that time taken to synchronize the network using optimally powered overheard nodes outdoes the existing designs in terms of performance parameters of the network such as throughput, network lifetime, end to end delay.

WSNs are unable to afford simultaneous transmission and reception of data and for most scenarios, the battery replacement is impossible upon the exhaustion of a node's battery energy. Thus, energy efficient protocols constitute vital design requirements for the WSN as a whole in order to increase the lifetime and ensure successful transmission of data from sensor node source to target, it becomes necessary to maintain sensor node's availability. Traveling Wave Pulse Coupled Oscillator (TWPCO) has proven to be robust, efficient and resistant to counteract deafness under various WSN including analytical models. However the extent to which energy efficient is consumed in sensor nodes, which deploys TWPCO as its self-organization has never been mentioned. To overcome this limitation, we performed a comparative study on energy efficient in TWPCO for WSNs. Using self-organizing scheme energy efficient WSNs by adopting a traveling wave biologically inspired network systems based on phase locking of Pulse Coupled Oscillator (PCO) model regards sensor nodes as observed in the flashing synchronization behaviors of fireflies and secretion of radio signals as firing. The simulation work was done using Java programming language. Energy efficiency in the random variant of both schemes (TWPCO and PCO) was also observed to be higher than the priority variant of the schemes.

Observations of natural phenomena are considered to be the best information source of spontaneous synchronization. Natural phenomena tend to match wireless sensor network (WSN) responses closely. Such synchronization is vital for the proper coordination of power cycles for energy conservation. A large number of fireflies employ the principle of pulse-coupled oscillators for light flash emission to attract mating partners. With respect to WSNs, the nodes are generally unable to afford packet transmission and reception simultaneously, thus preventing complete network synchronization. This paper presents a literature overview concerning the impact of the deafness problem on clock synchronization in a WSN. Data transmission based on synchronization can also be ensured through the optimization of energy usage periodic data capturing in a WSN. This study serves as a useful information source of clock synchronization to assist WSN researchers and novices in obtaining a better understanding of the impact of the deafness problem on clock synchronization and to enable them to promote effective designs and systems that address this problem.

 Abstract— A wireless sensor network is a distributed network system consisting of distributed nodes called sensors. Sensor nodes are disposed to to have errors. It is thus needed to sense and locate defective sensor nodes to ensure the quality of service of sensor networks. We propose an asynchronous based wakeup scheduling scheme, which optimizes energy saving ratio and achieves bounded neighbour discovery latency, without requiring time synchronization. Heterogeneous duty-cycling causes transmission latencies to be time-varying. Hence, the routing problem becomes more complex when the time domain must be considered for data delivery in duty-cycled WSNs. We formulate the routing problem as time-dependent Bellman-Ford problem. In this, it presents a multipath routing protocol for data transmission. The main issue is the energy wastage of unused nodes. So, to overcome this, it proposes an on demand asynchronous sleep awake protocol for reducing energy consumption. In this, optimal path selection is based on shortest distance between nodes which is to be calculated. The projected mechanism is implemented with MATLAB.

One of the most important characteristics of Wireless Sensor Networks is energy efficiency. Exhausted batteries cannot be easily replaced and cause connectivity problems and reduced network life. Therefore it is crucial to develop methods to reduce energy usage in those networks. In this work some methods in the MAC layer and network layer were tested in simulations for energy efficiency. The best combinations of methods were used in real implementation for confirmation.

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor's sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor's sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.

This paper describes an approach to the approaches being explored for a Sensor Network platform being developed for the DTI/NextWave technologies programme. The approach being adopted is to develop the system as a community of devices which use self-organising techniques to provide key functions. The devices are, largely, based on commodity technologies, thus providing a low cost basis. We give an outline of the approach and project and illustrate the techniques being developed with specific functions for: control, management, data retrieval and data quality control. The target application is off shore sea shelf monitoring; but the techniques being developed may be applied to a range of problems. enable the construction of cost effective sensor platforms, enabled by recent growth in wireless communications and microprocessor controlled consumer devices. This leads to small, low cost, limited functionality devices working in a cooperative networked mode, to form a single system. This model brings cost advantages and extra science value. By constructing a network of sensors, a greater area may be covered allowing better spatial resolution. Further, redundancy in components may be introduced thus providing more robust and longer lived operations. These features are highly advantages in applications covering large, hostile environments such as: glaciers, volcanoes or off-shore sea beds. This paper presents the development of such a sensor network (SN) platform for the monitoring of environmental impact on a coastal sea bed of a wind farm. Wind farms are seen as a key feature of the governments' sustainable energy policy. It is critical to ensure that their placement does not produce negative, environmental impacts. The complex interplay between the: oceans currents; wind; coast line and off shore features (e.g. sand banks) is poorly understood; but it is critical that natural costal defences such as sand banks are not disturbed or destroyed inadvertently. Thus there is a great need for continuous measurement of off shore features in such a context. This approach and application requires trade-off between design and capability. Using low cost devices entails limited functionality and performance. Target environments considered here are, by their nature, highly turbulent which entails that the sensor platforms have to be able to adjust to the environment on a continuous, though hard to predict, basis. These problems have led us to explore a new approach to the system engineering. In this approach the system is composed of a peer-to-peer system, supporting intrinsically networked algorithms; which derive their full functionality from being symmetric, cooperative and self-organising. This contrasts to current approaches of confining the functionality to individual nodes and considering the sensor network as a slave device. To achieve this, our approach is to model the algorithms on behaviours found in natural biological systems. This paper presents this overall concept and some example details of the algorithm design.

In a network of low-powered wireless sensors, it is essential to capture as many environmental events as possible while still preserving the battery life of the sensor node. This paper focuses on a real-time learning algorithm to extend the lifetime of a sensor node to sense and transmit environmental events. A common method that is generally adopted in ad-hoc sensor networks is to periodically put the sensor nodes to sleep. The purpose of the learning algorithm is to couple the sensor's sleeping behavior to the natural statistics of the environment hence that it can be in optimal harmony with changes in the environment, the sensors can sleep when steady environment and stay awake when turbulent environment. This paper presents theoretical and experimental validation of a reward based learning algorithm that can be implemented on an embedded sensor. The key contribution of the proposed approach is the design and implementation of a reward function that satisfies a trade-off between the above two mutually contradicting objectives, and a linear critic function to approximate the discounted sum of future rewards in order to perform policy learning.

Synchronicity is a useful abstraction in many sensor network applications. Communication scheduling, coordinated duty cycling, and time synchronization can make use of a synchronicity primitive that achieves a tight alignment of individual nodes' firing phases. In this paper we present the Reachback Firefly Algorithm (RFA), a decentralized synchronicity algorithm implemented on TinyOS-based motes. Our algorithm is based on a mathematical model that describes how fireflies and neurons spontaneously synchronize. Previous work has assumed idealized nodes and not considered realistic effects of sensor network communication, such as message delays and loss. Our algorithm accounts for these effects by allowing nodes to use delayed information from the past to adjust the future firing phase. We present an evaluation of RFA that proceeds on three fronts. First, we prove the convergence of our algorithm in simple cases and predict the effect of parameter choices. Second, we leverage the TinyOS simulator to investigate the effects of varying parameter choice and network topology. Finally, we present results obtained on an indoor sensor network testbed demonstrating that our algorithm can synchronize sensor network devices to within 100 µsec on a real multi-hop topology with links of varying quality.

In the full version of this paper [1] we present a reinforcement learning algorithm with the aim to increase the autonomous lifetime of a Wireless Sensor Network (WSN) and decrease latency in a decentralized manner. WSNs are collections of sensor nodes that gather environmental data, where the main challenges are the limited power supply of nodes and the need for decentralized control. To overcome these challenges, we make each sensor node adopt an algorithm to optimize the efficiency of a small group of surrounding nodes, so that in the end the performance of the whole system is improved. We compare our approach to conventional ad-hoc networks of different sizes and show that nodes in WSNs are able to develop an energy saving behaviour on their own and significantly reduce network latency, when using our reinforcement learning algorithm.

Abstract Observations of natural phenomena are considered to be the best information
source of spontaneous synchronization. Natural phenomena tend to match wireless sensor
network (WSN) responses closely. Such synchronization is vital for the proper coordination
of power cycles for energy conservation. A large number of fireflies employ the principle of
pulse-coupled oscillators for light flash emission to attract mating partners. With respect to
WSNs, the nodes are generally unable to afford packet transmission and reception

Synchronicity is a useful abstraction in many sensor network applications. Communication scheduling, coordinated duty cycling, and time synchronization can make use of a synchronicity primitive that achieves a tight alignment of individual nodes' firing phases. In this paper we present the Reachback Firefly Algorithm (RFA), a decentralized synchronicity algorithm implemented on TinyOS-based motes. Our algorithm is based on a mathematical model that describes how fireflies and neurons spontaneously synchronize. Previous work has assumed idealized nodes and not considered realistic effects of sensor network communication, such as message delays and loss. Our algorithm accounts for these effects by allowing nodes to use delayed information from the past to adjust the future firing phase. We present an evaluation of RFA that proceeds on three fronts. First, we prove the convergence of our algorithm in simple cases and predict the effect of parameter choices. Second, we leverage the TinyOS simulator to investigate the effects of varying parameter choice and network topology. Finally, we present results obtained on an indoor sensor network testbed demonstrating that our algorithm can synchronize sensor network devices to within 100 µsec on a real multi-hop topology with links of varying quality.

Synchronicity is a useful abstraction in many sensor network applications. Communication scheduling, coordinated duty cycling, and time synchronization can make use of a synchronicity primitive that achieves a tight alignment of individual nodes' firing phases. In this paper we present the Reachback Firefly Algorithm (RFA), a decentralized synchronicity algorithm implemented on TinyOS-based motes. Our algorithm is based on a mathematical model that describes how fireflies and neurons spontaneously synchronize. Previous work has assumed idealized nodes and not considered realistic effects of sensor network communication, such as message delays and loss. Our algorithm accounts for these effects by allowing nodes to use delayed information from the past to adjust the future firing phase. We present an evaluation of RFA that proceeds on three fronts. First, we prove the convergence of our algorithm in simple cases and predict the effect of parameter choices. Second, we leverage the TinyOS simulator to investigate the effects of varying parameter choice and network topology. Finally, we present results obtained on an indoor sensor network testbed demonstrating that our algorithm can synchronize sensor network devices to within 100 µsec on a real multi-hop topology with links of varying quality.

Recent experience with the deployment of sensor networks demonstrates that it is far from trivial to setup a working larger-scale sensor network in the field. Even though simulations and experiments with lab testbeds confirmed a working system, subtle real-world influences lead to frequent failures in the field. Identifying and fixing these problems in the field is currently a difficult and cumbersome task due to the lack of appropriate concepts and tools. In this paper we address this issue by, firstly, classifying common problems that have been encountered during deployment. We then show that many of these problems can be detected by overhearing and analyzing sensor network traffic without need for an instrumentation of sensor nodes. Based on this observation, we develop a tool to inspect a deployed sensor network, consisting of a distributed network sniffer and a data-stream-based framework for online traffic analysis. We demonstrate and evaluate how this tool can be used to debug a typical data gathering application. 1

Biologically inspired self-organization methods can help to manage the access control to the shared communication medium of wireless ad-hoc networks. One lightweight method is the primitive of desynchronization, which has already been implemented as MAC protocol for single-hop topologies successfully: periodically transmitting nodes establish a collision-free TDMA schedule autonomously. However, multi-hop topologies are more realistic, but each node now requires knowledge about its two-hop neighborhood to solve the hidden terminal problem.

Sensor networks are composed of nodes embedded in physical environments where applications may be tasked to run for years without human maintenance and without continuous external power supply. Strategies for power conservation are thus important in sensor network protocols and system architecture. One such strategy is to arrange node sleeping schedules so that radios are powered off until communication is necessary. Nodes cannot receive messages during periods when the radio is turned off. In this setting, there can arise situations where groups of network nodes have somehow become temporally partitioned : due to having different sleeping schedules, groups of nodes could be unaware of each other. The paper presents several self-stabilizing protocols to solve the problem of temporal partition; starting from an arbitrary temporally partitioned state, these protocols lead the network to a state in which all nodes have a perfectly aligned sleep schedule. Our techniques include using randomly chosen relatively prime sleep periods and occasional, and possibly random, probing of extra time slots. Our protocols aim for fast convergence while imposing only a small energy consumption overhead.

Abstract In this paper we propose SPARE MAC, a TDMA based medium access control (MAC) scheme for data diffusion in wireless sensor networks (WSNs). The rationale behind SPARE MAC is to spare energy through limiting the impact of idle listening and traffic overhearing.

Abstract In this paper we apply the COllective INtelligence (COIN) framework of Wolpert et al. to Wireless Sensor Networks (WSNs) with the aim to increase the autonomous lifetime of the network in a decentralized manner. COIN describes how selfish agents can learn to optimize their own performance, so that the performance of the global system is increased.