User demographics prediction based on mobile data

Social networking platforms such as Twitter provide an abundance of user generated content that is of interest to many sectors. Public opinion polling, targeted marketing, and business analytics are just a few applications that can benefit from this large amount of data. However, in order for some applications to make effective use of the data, author demographics are required. This project explores the techniques required to infer one highly sought-after demographic, namely age, treating the problem as a binary classification task of “under 30” and “over 30”. For classification, a Linear Support Vector Machine is employed, trained on features extracted from three publicly available user attributes, specifically, their description field, Tweets, and friends (the accounts that they are following). One of the major challenges of Twitter analysis is obtaining a high-quality dataset. For this project, an automated annotation technique is designed, using pattern matching to extract explicit declarations of a user’s age from their description field. From data collected over a one-week period, the approach was able to construct a dataset containing 62,450 users, with over 97% of the included users being correctly annotated. Although accurate, the approach was shown to result in some bias, most notably a large skew towards the younger age groups. The experiments explored various combinations of feature sets, and studied the effect that the size of the dataset has on classification performance. The results show that when used alone, friend based features perform competitively to Tweet based features, both obtaining very promising performances. Description based features also showed promise, however, due to the method of creating the dataset, it was found to be paramount to the generalisability of the model that any text used for annotation was not provided to the classifier. Overall, using a combination of all three user attributes obtained the best results, with over 86% classification performance. Due to Twitter’s API rate restrictions, current research has largely neglected using friends. However, the results from the dataset size experiments provide evidence that collecting a smaller number of users, but also including friends, may in fact be more effective than building a larger dataset that neglects them.

ACM SIGMOBILE Mobile Computing and Communications Review, 2014

Third party apps are an integral component of the smartphone ecosystem. In this paper, we investigate how user traits can be inferred by observing only a single snapshot of installed apps. Using supervised learning methods and minimal external information we show that user traits such as religion, relationship status, spoken languages, countries of interest, and whether or not the user is a parent of small children, can be easily predicted. Using data collected from over 200 smartphone users, specifically the list of installed apps and the corresponding ground truth traits of the users, we show that for most traits we can achieve over 90% precision. Our inference method can be used to provide services such as personalized content delivery or recommender systems for users. We also highlight privacy loss that can occur from unrestricted access to the app lists in popular smartphone operating systems.

Advances in Intelligent Systems and Computing, 2014

Demographic attributes such as gender and age of Internet users provide important information for marketing, personalization, and user behavior research. This paper addresses the problem of predicting users' gender based on browsing history. We employ a classification-based approach to the problem and investigate a number of features derived from browsing log data. We show that high-level content features such as topics or categories are very predictive of gender and combining such features with features derived from access times and browsing patterns leads to significant improvements in prediction accuracy. We empirically verified the effectiveness of the method on real datasets from Vietnamese online media. The method substantially outperformed a baseline, and achieved a macro-averaged F1 score of 0.805. Experimental results also demonstrate the effectiveness of combining different feature types: a combination of features achieved 12% improvement of F1 score over the best performing individual feature type.

User profile learning, such as mobility and demographic profile learning, is of great importance to various applications. Meanwhile, the rapid growth of multiple social platforms makes it possible to perform a comprehensive user profile learning from different views. However, the research efforts on user profile learning from multiple data sources are still relatively sparse, and there is no large-scale dataset released towards user profile learning. In our study, we contribute such benchmark and perform an initial study on user mobility and demographic profile learning. First, we constructed and released a large-scale multi-source multimodal dataset from three geographical areas. We then applied our proposed ensemble model on this dataset to learn user profile. Based on our experimental results, we observed that multiple data sources mutually complement each other and their appropriate fusion boosts the user profiling performance.

Proceedings of the 8th Workshop on Social Network Mining and Analysis - SNAKDD'14, 2014

We study the structure of the social graph of mobile phone users in the country of Mexico, with a focus on demographic attributes of the users (more specifically the users' age). We examine assortativity patterns in the graph, and observe a strong age homophily in the communications preferences. We propose a graph based algorithm for the prediction of the age of mobile phone users. The algorithm exploits the topology of the mobile phone network, together with a subset of known users ages (seeds), to infer the age of remaining users. We provide the details of the methodology, and show experimental results on a network GT with more than 70 million users. By carefully examining the topological relations of the seeds to the rest of the nodes in GT , we find topological metrics which have a direct influence on the performance of the algorithm. In particular we characterize subsets of users for which the accuracy of the algorithm is 62% when predicting between 4 age categories (whereas a pure random guess would yield an accuracy of 25%). We also show that we can use the probabilistic information computed by the algorithm to further increase its inference power to 72% on a significant subset of users.

ArXiv, 2021

These days, due to the increasing amount of information generated on the web, most web service providers try to personalize their services. Users also interact with web-based systems in multiple ways and state their interests and preferences by rating the provided items. In this paper, we propose a framework to predict users’ demographic based on ratings registered by users in a system. To the best of our knowledge, this is the first time that the item ratings are employed for users’ demographic prediction problem, which has extensively been studied in recommendation systems and service personalization. We apply the framework to Movielens dataset’s ratings and predict users’ age and gender. The experimental results show that using all ratings registered by users improves the prediction accuracy by at least 16% compared with previously studied models. Moreover, by classifying the items as popular and unpopular, we eliminate ratings belong to 95% of items and still reach an acceptable...

Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, 2015

User profile learning, such as mobility and demographic profile learning, is of great importance to various applications. Meanwhile, the rapid growth of multiple social platforms makes it possible to perform a comprehensive user profile learning from different views. However, the research efforts on user profile learning from multiple data sources are still relatively sparse, and there is no large-scale dataset released towards user profile learning. In our study, we contribute such benchmark and perform an initial study on user mobility and demographic profile learning. First, we constructed and released a large-scale multi-source multimodal dataset from three geographical areas. We then applied our proposed ensemble model on this dataset to learn user profile. Based on our experimental results, we observed that multiple data sources mutually complement each other and their appropriate fusion boosts the user profiling performance.

Journal of Big Data, Springer. [Rank: Q1], 2019

Due to the increasing popularity of recent advanced features and context-awareness in smart mobile phones, the contextual data relevant to users’ diverse activities with their phones are recorded through the device logs. Modeling and predicting individual’s smartphone usage based on contexts, such as temporal, spatial, or social information, can be used to build various context-aware personalized systems. In order to intelligently assist them, a machine learning classifier based usage prediction model for individual users’ is the key. Thus, we aim to analyze the effectiveness of various machine learning classification models for predicting personalized usage utilizing individual’s phone log data. In our context-aware analysis, we first employ ten classic and wellknown machine learning classification techniques, such as ZeroR, Naive Bayes, Decision Tree, Random Forest, Support Vector Machine, K-Nearest Neighbors, Adaptive Boosting, Repeated Incremental Pruning to Produce Error Reduction, Ripple Down Rule Learner, and Logistic Regression classifiers. We also present the empirical evaluations of Artificial Neural Network based classification model, which is frequently used in deep learning and make comparative analysis in our context-aware study. The effectiveness of these classifier based context-aware models is examined by conducting a range of experiments on the real mobile phone datasets collected from individual users. The overall experimental results and discussions can help both the researchers and applications developers to design and build intelligent context-aware systems for smartphone users.

Computer Networks

Mobile users demand more and more data traffic, yet network resources are limited. This creates a challenge for network resource management. One way of addressing this challenge is by understanding the data usage patterns of mobile users so that resources can be optimally allocated based on user traffic demand and data usage behavior. However, understanding and characterizing the data usage patterns of mobile users is a complex task. In this work, we investigate and characterize users' data usage patterns and behavior in mobile networks. We leverage a dataset (∼113 million records) collected through a crowd-based mobile network measurement platform-Netradar-across five countries. Data usage behavior of users over a cellular network is primarily driven by user mobility, the type of subscription plan marketed by Mobile Network Operators (MNOs), network congestion, and network coverage. We apply an unsupervised machine learning approach to cluster mobile user types by considering different factors such as data consumption, network access type, the number of sessions created per user, throughput, and mobility. By defining data usage pattern of mobile users, we develop a user clustering model and identify three different mobile user groups (clusters). Our clustering model shows that the data usage patterns are unevenly distributed across the five countries studied, characterized by a small number of heavy users consuming the highest volume of data. We show how the types of applications installed by users correlate with data consumption patterns in some countries. Heavy users tend to install more traffic-demanding apps than users from the other two groups-regular and light users. Finally, we trained a classification model using the labeled dataset produced by our aforementioned user clustering method. The model helps classifying mobile users according to their usage patterns (i.e., heavy, regular, and light) with an accuracy of ∼80% in the test dataset.

2015

Interacting with smart devices is a common experience and is becoming an integral part of daily life for many people. Modern smart devices are equipped with a large variety of environmental and user input sensors. We hypothesize that a fusion of smart device sensor data can provide biometric data that allows for classification of user demographics such as age, gender, and native language. A smart device is instrumented with sensor data collection software and with user demographic classification software. An experiment is devised where data is collected for a sample group of users. The data is analyzed, and two classification algorithms are implemented based on fusion of the different sensors. The classification methods are based upon decision tree and principle component analysis. The results of the experiment indicate that high accuracy is achieved for user demographic classification. Finally, we further discuss the applications and limitations.