Miguel Pardal

Universidade de Lisboa, IST - Instituto Superior Técnico, Faculty Member

Followers

Following

Co-authors

Public Views

I've been a researcher and lecturer at the Computer Science and Engineering Department of Instituto Superior Técnico, Universidade Técnica de Lisboa since 2002, teaching distributed systems, enterprise applications integration and operating systems courses.

Before my return to academia, I worked from 2000 to 2002 as a consultant for Unisys, in major Banking and Insurance projects. My work was recognized in 2004, when I was awarded the best computer science admission work to the Portuguese Engineers Association (Ordem dos Engenheiros).

I've authored 6 papers. My main research interests include web services, enterprise architectures, integration technologies, RFID and sensor networks.
Supervisors: José Alves Marques and Sanjay Sarma

less

Interests

Uploads

Papers by Miguel Pardal

Does Big Data Require Complex Systems? A Performance Comparison Between Spark and Unicage Shell Scripts

arXiv (Cornell University), Dec 27, 2022

The paradigm of big data is characterized by the need to collect and process data sets of great v... more The paradigm of big data is characterized by the need to collect and process data sets of great volume, arriving at the systems with great velocity, in a variety of formats. Spark is a widely used big data processing system that can be integrated with Hadoop to provide powerful abstractions to developers, such as distributed storage through HDFS and resource management through YARN. When all the required configurations are made, Spark can also provide quality attributes, such as scalability, fault tolerance, and security. However, all of these benefits come at the cost of complexity, with high memory requirements, and additional latency in processing. An alternative approach is to use a lean software stack, like Unicage, that delegates most control back to the developer. In this work we evaluated the performance of big data processing with Spark versus Unicage, in a cluster environment hosted in the IBM Cloud. Two sets of experiments were performed: batch processing of unstructured data sets, and query processing of structured data sets. The input data sets were of significant size, ranging from 64 GB to 8192 GB in volume. The results show that the performance of Unicage scripts is superior to Spark for search workloads like grep and select, but that the abstractions of distributed storage and resource management from the Hadoop stack enable Spark to execute workloads with inter-record dependencies, such as sort and join, with correct outputs.

Download

MultiTLS: using multiple and diverse ciphers for stronger secure channels

Computers & Security, Jun 1, 2023

Sensmart

Currently, there are millions of sensors connected to the Internet. These sensors gather various ... more Currently, there are millions of sensors connected to the Internet. These sensors gather various types of data, from temperature, humidity, sound and image, to location or biometrics, to name a few. These kinds of data can be very relevant for scientific or business purposes. However, there is no online platform or marketplace where it can be easily exchanged. In this work we design and implement Sensmart, a solution through which it is possible to purchase and sell sensor data. Suppliers share their devices or data and consumers can buy data or acquire control of a device over a period of time. Sensmart goes beyond data exchange, and provides the ability to control a sensing device, for example, a customer can position a camera or move a robot. The platform was tested and evaluated through use cases and the implemented solution allows customers to share sensor devices and the data in an effective way. CCS CONCEPTS • Hardware → Sensor devices and platforms; • Information systems → Data access methods; • Computer systems organization → Sensor networks;

Download

Smart Healthcare Monitoring System For Healthy Driving in Public Transportation

2023 18th Iberian Conference on Information Systems and Technologies (CISTI)

Automatic Generation of Distributed Algorithms with Generative AI

2023 53rd Annual IEEE/IFIP International Conference on Dependable Systems and Networks - Supplemental Volume (DSN-S)

Synthesis of Fault-Tolerant Reliable Broadcast Algorithms With Reinforcement Learning

IEEE Access

Fault-tolerant algorithms, such as Reliable Broadcast, assure the correct operation of modern dis... more Fault-tolerant algorithms, such as Reliable Broadcast, assure the correct operation of modern distributed systems, even when some of the system nodes fail. However, the development of distributed algorithms is a manual and complex process, where slight changes in requirements can require a complete redesign of the algorithm. To automate the process of developing such algorithms, this work presents a new approach that uses Reinforcement Learning to synthesize correct and efficient fault-tolerant distributed algorithms. This work shows the first application of the approach on the synthesis of fault-tolerant Reliable Broadcast algorithms. The presented technique is capable of synthesizing correct and efficient algorithms with the same performance as others available in the literature as well as a new Byzantine tolerant algorithm, in only 12, 000 learning episodes. Based on the success of this implementation, we aim, in the future, to extend this technique to other distributed algorithms such as Consensus. INDEX TERMS Fault-tolerant distributed algorithms, reliable broadcast, program synthesis, reinforcement learning, program verification, model-checking.

Download

ComSEC: Secure Communications for Baggage Handling Systems

Lecture Notes in Computer Science, 2023

Smart meter data processing: a showcase for simple and efficient textual processing

arXiv (Cornell University), Dec 27, 2022

The increase in the production and collection of data from devices is an ongoing trend due to the... more The increase in the production and collection of data from devices is an ongoing trend due to the roll-out of more cyber-physical applications. Smart meters, because of their importance in power grids, are a class of such devices whose produced data requires meticulous processing. In this paper, we use Unicage, a data processing system based on classic Unix shell scripting, that delivers excellent performance in a simple package. We use this methodology to process smart meter data in XML format, subjected to the constraints posed by a real use case. We develop a solution that parses, validates and performs a simple aggregation of 27 million XML files in less than 10 minutes. We present a study of the solution as well as the benefits of its adoption.

Download

Learning to generate Reliable Broadcast Algorithms

arXiv (Cornell University), Jul 31, 2022

Modern distributed systems are supported by fault-tolerant algorithms, like Reliable Broadcast an... more Modern distributed systems are supported by fault-tolerant algorithms, like Reliable Broadcast and Consensus, that assure the correct operation of the system even when some of the nodes of the system fail. However, the development of distributed algorithms is a manual and complex process, resulting in scientific papers that usually present a single algorithm or variations of existing ones. To automate the process of developing such algorithms, this work presents an intelligent agent that uses Reinforcement Learning to generate correct and efficient fault-tolerant distributed algorithms. We show that our approach is able to generate correct fault-tolerant Reliable Broadcast algorithms with the same performance of others available in the literature, in only 12, 000 learning episodes.

Download

MIRES: Intrusion Recovery for Applications Based on Backend-As-a-Service

IEEE Transactions on Cloud Computing

μVerum: Intrusion Recovery for Microservice Applications

IEEE Access

Microservice architectures allow complex applications to be developed as a collection of loosely ... more Microservice architectures allow complex applications to be developed as a collection of loosely coupled components. The heterogeneous architecture of these applications makes the process of recovering from intrusions especially complex, error-prone, and time-consuming. Although there are several recovery mechanisms for monolithic applications, applying such mechanisms in microservices would not work due to the distribution of the components, the different technologies used by each service, and their scale. Moreover, it can be difficult to trace the services affected by an intrusion and which actions to revert. We propose µVerum, a framework for recovering microservices from intrusions that corrupt the application state. Our approach allows recovery of large-scale microservice applications by logging user requests and the operations that are propagated through several microservices. When a system administrator detects a faulty request, µVerum can execute compensating operations in each of the affected microservices. We implemented, evaluated, and made the code of µVerum available. Our experiments show that µVerum is able to revert the effects in an intrusion in one second while the application is running. INDEX TERMS Microservices, cloud computing, intrusion recovery.

Download

SureRepute: Reputation System for Crowdsourced Location Witnesses

2022 IEEE 21st International Symposium on Network Computing and Applications (NCA)

LoCaaS: Location-Certification-as-a-Service

2022 IEEE 21st International Symposium on Network Computing and Applications (NCA)

Bluetooth Peer-to-Peer Location Certification with a Gamified Mobile Application

2022 IEEE 21st International Symposium on Network Computing and Applications (NCA)

Lisbon Hotspots: Wi-Fi access point dataset for time-bound location proofs

Cornell University - arXiv, Aug 5, 2022

Wi-Fi hotspots are a valuable resource for people on the go, especially tourists, as they provide... more Wi-Fi hotspots are a valuable resource for people on the go, especially tourists, as they provide a means to connect personal devices to the Internet. This extra connectivity can be helpful in many situations, e.g., to enable map and chat applications to operate outdoors when cellular connectivity is unavailable or is expensive. Retail stores and many public services have recognized that hotspots have potential to attract and retain customers, so many of them offer free and open Wi-Fi. In busy cities, with many locals and visitors, the number of hotspots is very significant. Some of these hotspots are available for long periods of time, while others are short-lived. When we have many users with devices collecting hotspot observations, they can be used to detect the location-using the long-lived hotspots-and to prove the time when the location was visited-using the short-lived hotspots observed by others users at the location. In this article, we present a dataset of collected Wi-Fi data from the most important tourist locations in the city of Lisbon, Portugal, over a period of months, that was used to show the feasibility of using hotspot data for location detection and proof. The obtained data and algorithms were assessed for a specific use case: smart tourism. We also present the data model used to store the observations and the algorithms developed to detect and prove location of a user device at a specific time. The Lisbon Hotspots dataset, LXspots, is made publicly available to the scientific community so that other researchers can also make use of it to develop new and innovative mobile and Internet of Things applications.

Download

A Study on the Best Way to Compress Natural Language Processing Models

2022 IEEE International Conference on Fuzzy Systems (FUZZ-IEEE)

CROSS City: Wi-Fi Location Proofs for Smart Tourism

Ad-Hoc, Mobile, and Wireless Networks, 2020

Sanare: Pluggable Intrusion Recovery for Web Applications

Web applications are exposed to many threats and, despite the best defensive efforts, are often s... more Web applications are exposed to many threats and, despite the best defensive efforts, are often successfully attacked. Reverting the effects of an attack on the state of such an application requires a profound knowledge about the application, to understand what data did the attack corrupt. Furthermore, it requires knowing what steps are needed to revert the effects without modifying legitimate data created by legitimate users. Existing intrusion recovery systems are capable of reverting the effects of the attack but they require modifications to the source code of the application, which may be unpractical. We present Sanare, a pluggable intrusion recovery system designed for web applications that use different data storage systems to keep their state. Sanare does not require any modification to the source code of the application or the web server. Instead, it uses Matchare, a new deep learning scheme we introduce to learn the matches between the HTTP requests and the database statem...

Download

STOP: A Location Spoofing Resistant Vehicle Inspection System

Ad-Hoc, Mobile, and Wireless Networks, 2020

Big Data Privacy by Design Computation Platform

We live in the age of Big Data, and personal user data, in particular, is necessary for the opera... more We live in the age of Big Data, and personal user data, in particular, is necessary for the operation and improvement of healthcare services. Many times, the capture and use of personal data are not made explicit to the users, but they are central to the business model of companies. However, each person’s right to privacy needs to be respected.

Download

Miguel Pardal

Uploads

Papers by Miguel Pardal

Log In