Faces in the Distorting Mirror

The capabilities of modern devices, coupled with the almost ubiquitous availability of Internet connectivity, have resulted in photos being shared online at an unprecedented scale. This is further amplified by the popularity of social networks and the immediacy they offer in content sharing. Existing access control mechanisms are too coarse-grained to handle cases of conflicting interests between the users associated with a photo; stories of embarrassing or inappropriate photos being widely accessible have become quite common. In this paper, we propose to rethink access control when applied to photos, in a way that allows us to effectively prevent unwanted individuals from recognizing users in a photo. The core concept behind our approach is to change the granularity of access control from the level of the photo to that of a user's personally identifiable information (PII). In this work, we consider the face as the PII. When another user attempts to access a photo, the system determines which faces the user does not have the permission to view, and presents the photo with the restricted faces blurred out. Our system takes advantage of the existing face recognition functionality of social networks, and can interoperate with the current photo-level access control mechanisms. We implement a proof-of-concept application for Facebook, and demonstrate that the performance overhead of our approach is minimal. We also conduct a user study to evaluate the privacy offered by our approach, and find that it effectively prevents users from identifying their contacts in 87.35% of the restricted photos. Finally, our study reveals the misconceptions about the privacy offered by existing mechanisms, and demonstrates that users are positive towards the adoption of an intuitive, straightforward access control mechanism that allows them to manage the visibility of their face in published photos.

Facebook has launched a facial recognition feature and incorporated it by default for all users. This feature is also known as Photo Tag Suggest. It enables the automatic scanning of faces on newly uploaded photos and matches these faces with new photos with Facebook users on existing photos. To the best of our knowledge, Facebook has never published any information about the technologies that this feature uses and, therefore, we have investigated the possible technologies used for a feature seen as this. Facebook's facial recognition feature associates tags with the accounts of other Facebook users. This is most likely done by comparing different faces within their database and by also taking their social contexts into account. This paper also reviews privacy concerns relating to Facebook's facial recognition features based on a research conducted amongst students using Facebook and also presents guidelines and suggestions for increasing privacy awareness.

Journal of Experimental Psychology: Applied, 2014

Viewers find it difficult to match photos of unfamiliar faces for identity. Despite this, the use of photographic ID is widespread. In this study we ask whether it is possible to improve face matching performance by replacing single photographs on ID documents with multiple photos or an average image of the bearer. In 3 experiments we compare photo-to-photo matching with photo-to-average matching (where the average is formed from multiple photos of the same person) and photo-to-array matching (where the array comprises separate photos of the same person). We consistently find an accuracy advantage for average images and photo arrays over single photos, and show that this improvement is driven by performance in match trials. In the final experiment, we find a benefit of 4-image arrays relative to average images for unfamiliar faces, but not for familiar faces. We propose that conventional photo-ID format can be improved, and discuss this finding in the context of face recognition more generally.

— Photo sharing is an attractive feature which popularizes Online Social Networks (OSNs). Unfortunately, it may leak users' privacy if they are allowed to post, comment, and tag a photo freely. In this paper, we attempt to address this issue and study the scenario when a user shares a photo containing individuals other than himself/herself (termed co-photo for short). To prevent possible privacy leakage of a photo, we design a mechanism to enable each individual in a photo be aware of the posting activity and participate in the decision making on the photo posting. For this purpose, we need an efficient facial recognition (FR) system that can recognize everyone in the photo. However, more demanding privacy setting may limit the number of the photos publicly available to train the FR system[1]. To deal with this dilemma, our mechanism attempts to utilize users' private photos to design a personalized FR system specifically trained to differentiate possible photo co-owners without leaking their privacy. We also develop a distributed consensus based method to reduce the computational complexity and protect the private training set. We show that our system is superior to other possible approaches in terms of privacy using encryption algorithm and opensource. Our mechanism is implemented as a proof of concept Android application on Face book's platform.

While Online Social Networks (OSNs) enable users to share photos easily, they also expose users to several privacy threats from both the OSNs and external entities. The current privacy controls on OSNs are far from adequate, resulting in inappropriate flows of information when users fail to understand their privacy settings or OSNs fail to implement policies correctly. OSNs may further complicate privacy expectations when they reserve the right to analyze uploaded photos using automated face identification techniques.

Applied Sciences

Social media has now become a part of people’s lives. That is, today people interact on social media in a way that never happened before, and its important feature is to share photos and events with friends and family. However, there are risks associated with posting pictures on social media by unauthorized users. One of these risks is the privacy violation, where the published pictures can reveal more details and personal information. Since this issue has not yet investigated, this paper thus evaluates a methodology to address this issue, which is a precedent of its kind. In particular, our methodology relies on effective systems for detecting faces and recognizing faces in published images using facial recognition techniques. To evaluate the proposed idea, we developed an application using convolutional neural network (CNN) and the results showed that the proposed methodology can protect privacy and reduce its violation on online social networks.

2011

Secure facial identification systems compare an input face to a protected list of subjects. If this list were to be made public, there would be a severe privacy/confidentiality breach. A common approach to protect these lists of faces is to store a representation (descriptor or vector) of the face that is not directly mappable to its original form. In this thesis, we consider a recently developed secure identification system, Secure Computation of Face Identification (SCiFI) [1], that utilizes an index-based facial vector to discretely compress the representation of a face image. A facial descriptor of this system does not allow for a complete reverse mapping. However, we show that if a malicious user is able to obtain a facial descriptor, it is possible that he/she can reconstruct an identifiable human face. We develop a novel approach to initially assemble the information given by the SCiFI protocol to create a fragmented face. This image has large missing regions due to SCiFI's facial representation. Thus, we estimate the missing regions of the face using an iterative Principal Component Analysis (PCA) technique. This is done by first building a face subspace based on a public set of human faces. Then, given the assembled image from the SCiFI protocol, we iteratively project this image in and out of the subspace to obtain a complete human face. Traditionally, PCA reconstruction techniques have been used to estimate very small or specific occluded regions of a face image; these techniques have also been used in facial recognition such as through a k-nearest neighbor approach. However, in our new method, we use it to synthesize 60% to 80% of a human face for facial identification. We explore novel methods of comparing images from different subspaces using metric learning and other forms of facial descriptors. We test our reconstruction with face identification tasks given to a human and a computer. Our results show that our reconstructions are consistently more informative than what is extracted from the SCiFI facial descriptor alone. In addition, these tasks show that our reconstructions are identifiable by humans and computers. The success of our approach implies that a malicious attacker could expose the faces on a registered database.

2013 IEEE Workshop on Applications of Computer Vision (WACV), 2013

Secure Computation of Face Identification (SCiFI) [20] is a recently developed secure face recognition system that ensures the list of faces it can identify (e.g., a terrorist watch list) remains private. In this work, we study the consequences of malformed input attacks on the system-from both a security and computer vision standpoint. In particular, we present 1) a cryptographic attack that allows a dishonest user to undetectably obtain a coded representation of faces on the list, and 2) a visualization approach that exploits this breach, turning the lossy recovered codes into human-identifiable face sketches. We evaluate our approach on two challenging datasets, with face identification tasks given to a computer and human subjects. Whereas prior work considered security in the setting of honest inputs and protocol execution, the success of our approach underscores the risk posed by malicious adversaries to todays automatic face recognition systems.

IEEE Transactions on Information Forensics and Security, 2015

Widespread deployment of biometric systems supporting consumer transactions is starting to occur. Smart consumer devices, such as tablets and phones, have the potential to act as biometric readers authenticating user transactions. However, the use of these devices in uncontrolled environments is highly susceptible to replay attacks where these biometric data are captured and replayed at a later time. Current approaches to counter replay attacks in this context are inadequate. In order to show this, we demonstrate a simple replay attack that is 100% effective against a recent state-of-the-art face recognition system; this system was specifically designed to robustly distinguish between live people and spoofing attempts such as photographs. This paper proposes an approach to counter replay attacks for face recognition on smart consumer devices using a noninvasive challenge and response technique. The image on the screen creates the challenge, and the dynamic reflection from the person's face as they look at the screen forms the response. The sequence of screen images and their associated reflections digitally watermarks the video. By extracting the features from the reflection region, it is possible to determine if the reflection matches the sequence of images that were displayed on the screen. Experiments indicate that the face reflection sequences can be classified under ideal conditions with a high degree of confidence. These encouraging results may pave the way for further studies in the use of video analysis for defeating biometric replay attacks on consumer devices.

2014

Biometric systems are generally restricted to specialist deployments and require expensive equipment. However, the world has recently experienced the widespread rollout of cheap biometric devices in the form of smart phones and tablets. One of the main drivers for mainstream adoption of biometric technologies is the need to address continuing problems with authenticating to online systems. These mobile devices may now be suitable to provide biometric-based authentication to a wide user population. This chapter discusses the different ways that face recognition can be used on smart mobile devices. The authors highlight the online authentication problem and show how three-factor authentication can address many pressing issues. They also discuss the ways that such a system could be attacked, and focus on replay attacks which have yet to be seriously addressed in the literature. The authors conclude with a brief examination of the current research into addressing replay attacks.