Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Related Works
Scope of the Problem
Pre-Processing the Data
Clustering the Data
Preliminary Analysis
Conclusion
Future Scope for Improvement
References

Final Documentation

Profile image of Kaushik GVSNKaushik GVSN

Abstract
sparkles

AI

This documentation covers the integration of various clustering techniques, specifically K-Means, Gaussian Mixture Models (GMM), and HDBSCAN, to analyze and classify data derived from tweets. The approach incorporates dimensionality reduction through Word2Vec and tf-idf measures, leading to effective data representation. The paper highlights the effectiveness of these methodologies using metrics such as Silhouette Score, Calinski-Harabasz Index, and Davies-Bouldin Index to evaluate the clustering performance.

Related papers

Proceedings of the WESS'15: Workshop on Embedded Systems Security
Stavros Koubias

2015

View PDFchevron_right
Embedded Systems: Security Threats and Solutions
American journal of Engineering Research (AJER)

With the increasing use of embedded devices in our daily life, security threats have also been increasing in a proportional rate. However, ensuring security in the embedded systems has become a great challenge not only for the embedded device experts but also for the manufacturers. The problem especially arises because of the limited hardware and software implementation options for the designers. At the same time, companies are trying to keep the vulnerabilities of the operating system of those embedded devices in secret and they are not relieving any necessary security updates quickly. It has become very urgent to ensure proper security of the embedded systems to save it from any major technological disaster near future. In this paper, we have broadly discussed the structures, characteristics and applications of different embedded devices in our daily life. Beside this, we have also discussed about the different causes of security threats and some of our suggested solutions to protect the systems from the attackers as well that we have found in our research

View PDFchevron_right
SECURITY AND INTEGRITY ASPECTS AND APPROACHES IN INTERNET OF THINGS
Dr S kiran

The IoT, refers to the billions of physical things across the world currently connected to the Internet. Thanks to the advent of computer chips and the availability of wireless networks, something as little as a pill may be transformed into a part of an IoT to become something as large as an airplane. Connecting and adding sensors to all these various items provides a degree of digital intelligence to equipment that would otherwise be stupid so that it can transmit in real time information without engaging a person. The Internet of Things enhances and responsives the world around us by fusing the digital and physical realities. A complete IoT safety portfolio provides developers with the opportunity to secure devices from every kind of vulnerability while implementing the safety level most suited to their application demands. Cryptography technologies assist fight assaults, while security can protect them from assaults on the life cycle. Isolation precautions may be used to prevent software attacks, while mitigation and side-channel attack mitigation technologies are vital for combating physical chip assaults.

View PDFchevron_right
Securing Access to Embedded Systems - An Effective Concept for Devices Lacking Internet Connection
Bruno Juchli

Many embedded systems provide a web interface for maintenance tasks such as system configuration, test execution and firmware updating. Access to this interface usually needs to be restricted to authorized employees. This paper shows an efficient and cost-effective concept to secure maintenance interfaces using widespread standards and technology. By storing authorisation information in standard compliant X.509 certificate extensions Transport Layer Security (TLS) and X.509 Public Key Infrastructure (PKI) provide mutual authentication, message integrity as well as confidentiality and enable authorisation of employees. Practical experience of the implementation completes this paper.

View PDFchevron_right
Embedded system security
Thiru Neels

Computer, 2004

View PDFchevron_right
Analysis and Prevention of Security Vulnerabilities in a Smart City
Jay Thom

2022 IEEE 12th Annual Computing and Communication Workshop and Conference (CCWC), 2022

In recent years, there has been a growing interest in so-called smart cities. These cities use technology to connect and enhance the lives of their citizens. Smart cities use many Internet of Things (IoT) devices, such as sensors and video cameras, that are interconnected to provide constant feedback and up-to-date information on everything that is happening. Despite the benefits of these cities, they introduce a numerous new vulnerabilities as well. These smart cities are now susceptible to cyber-attacks that aim to "alter, disrupt, deceive, degrade, or destroy computer systems." Through the use of an educational and research-based IoT test-bed with multiple networking layers and heterogeneous devices connected to simultaneously support networking research, anomaly detection, and security principles, we can pinpoint some of these vulnerabilities. This work will contribute potential solutions to these vulnerabilities that can hopefully be replicated in smart cities around the world. Specifically, in the transportation section of our educational smart city several vulnerabilities in the signal lights, street lights, and the cities train network were discovered. To conduct this research two scenarios were developed. These consisted of inside the network security and network perimeter security. For the latter we were able to find extensive vulnerabilities that would allow an attacker to map the entire smart city sub-network. Solutions to this problem are outlined that utilize an Intrusion Detection System and Port Mirroring. However, while we were able to exploit the city's Programmable Logic Controller (PLC) once inside the network, it was found that due to dated Supervisory Control and Data Acquisition (SCADA) systems, there were almost no solutions to these exploits.

View PDFchevron_right
A Verified Architecture for Proofs of Execution on Remote Devices under Full Software Compromise
Karim Eldefrawy

ArXiv, 2020

Modern society is increasingly surrounded by, and accustomed to, a wide range of Cyber-Physical Systems (CPS), Internet-of-Things (IoT), and smart devices. They often perform safety-critical functions, e.g., personal medical devices, automotive CPS and industrial automation (smart factories). Some devices are small, cheap and specialized sensors and/or actuators. They tend to run simple software and operate under control of a more sophisticated central control unit. The latter is responsible for the decision-making and orchestrating the entire system. If devices are left unprotected, consequences of forged sensor readings or ignored actuation commands can be catastrophic, particularly, in safety-critical settings. This prompts the following three questions: (1) How to trust data produced by a simple remote embedded device? and (2) How to ascertain that this data was produced via execution of expected software? Furthermore, (3) Is it possible to attain (1) and (2) under the assumptio...

View PDFchevron_right
Achieving Data Dissemination with Security using FIWARE and Intel Software Guard Extensions (SGX)
Dalton Valadares

2018 IEEE Symposium on Computers and Communications (ISCC), 2018

The Internet of Things (IoT) field has gained much attention from industry and academia, being the main subject for numerous research and development projects. Frequently, the dense amount of generated data from IoT applications is sent to a cloud service, that is responsible for processing and storage. Many of these applications demand security and privacy for their data because of their sensitive nature. This is specially true when such data must be processed in entities hosted in public clouds, where the environment in which applications run may not be trusted. Some concerns are then raised since it is not trivial to provide the needed protection for these sensitive data. We present a solution that considers the security components of FIWARE and the Intel SGX capabilities. FIWARE is a platform created to support the development of Smart Applications, including IoT systems, and SGX is the Intel solution for Trusted Execution Environment (TEE). We propose a new component for key management that, together with other FIWARE components, can be used to provide privacy, confidentiality, and integrity guarantees for IoT data. A case study illustrates how this proposed solution can be employed in a realistic scenario, which allows the dissemination of sensitive data through public clouds without risking privacy issues. The results of the experiments provide evidence that our approach does not harm scalability or availability of the system. In addition, it presents acceptable memory costs when considering the benefit of the privacy guarantees achieved. The Internet of Things (IoT) concept has gained considerable importance in industry and academia in the last years. The possibility of interconnecting a huge number of devices allows the creation of various new applications. This is possible thanks to the advances in devices technologies and communication protocols that enable such interconnection. IoT sensors and actuators are increasingly becoming smarter

View PDFchevron_right
Secure Controls for Smart Cities; Applications in Intelligent Transportation Systems and Smart Buildings
Zahra Alisha

International Journal of Science and Engineering Applications, 2019

Internet of Things (IoT) is an emerging concept in smart infrastructures describing a wide ecosystem where interconnected devices and services collect, exchange and process data in order to adapt dynamically to a context. IoT is tightly bound to cyber-physical systems and in this respect is an enabler of Smart Infrastructures by enhancing their quality of service provisioning. Smart infrastructure, enabled by technologies like IoT, offer numerous of advantages bringing serious cost savings and efficiencies. These kinds of data-driven environments, fueled by connected devices and network connectivity, become a new attack surface for cyber threats. Cyber security approaches develop guidance to secure IoT and Smart Infrastructures from cyber threats, by highlighting good security practices and proposing recommendations to operators, manufacturers and decision makers. In this paper, cyber security approaches are analyzed in a smart city infrastructure.

View PDFchevron_right
DIAT: Data Integrity Attestation for Resilient Collaboration of Autonomous Systems
tigist Abera

Proceedings 2019 Network and Distributed System Security Symposium

View PDFchevron_right

References (41)

  1. Lastly, we see that the noise contains almost an amalgamation of a number of residual artifacts from previous clusters. Based on these wordclouds, we can identify 4 Major Categories of traffic related inci- dents:
  2. Breakdown related: Cluster 3
  3. Flyover related (can be a part of breakdowns): Cluster 1
  4. City programs: Clusters 2 and 0
  5. Road blockade/diversions: Cluster 4 References
  6. Szymanski & Ciota, 2002)Szymanski, G., & Ciota, Z. (2002). Hidden Markov Models Suitable for Text Generation. WSEAS International Conference on Signal, Speech and Image Processing (WSEAS ICOSSIP 2002), 1, 3081-3084.
  7. Parikh, 2012)Parikh, R. (2012). ET : Events from Tweets. 613-620.
  8. Lofi & Krestel, 2012)Lofi, C., & Krestel, R. (2012). iParticipate: Automatic tweet generation from local government data. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7239 LNCS(PART 2), 295-298. https://doi.org/10.1007/978-3-642-29035-0_24
  9. Tapiador, Carrera, & Salvachúa, 2012)Tapiador, A., Carrera, D., & Salvachúa, J. (2012). Social stream, a social network framework. 1st International Conference on Future Generation Communication Technologies, FGCT 2012, 52-57. https://doi.org/10.1109/FGCT.2012.6476557
  10. Sakaki, T., Okazaki, M., & Matsuo, Y. (2013). Tweet analysis for real-time event detection and earthquake reporting system development. IEEE Transactions on Knowledge and Data Engineering, Vol. 25, pp. 919-931. https://doi.org/10.1109/TKDE.2012.29
  11. Campello, Moulavi, & Sander, 2013)Campello, R. J. G. B., Moulavi, D., & Sander, J. (2013). Density-based clustering based on hierarchical density estimates. Lecture Notes in Computer Science (Including Subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 7819 LNAI(PART 2), 160-172. https://doi.org/10.1007/978-3-642-37456-2_14
  12. Lloret & Palomar, 2013)Lloret, E., & Palomar, M. (2013). Towards automatic tweet generation: A comparative study from the text summarization perspective in the journalism genre. Expert Systems with Applications, 40(16), 6624-6630. https://doi.org/10.1016/j.eswa.2013.06.021
  13. Rosi, Mamei, & Zambonelli, 2013)Rosi, A., Mamei, M., & Zambonelli, F. (2013). Integrating social sensors and pervasive services: Approaches and perspectives. International Journal of Pervasive Computing and Communications, 9(4), 294-310. https://doi.org/10.1108/IJPCC-09- 2013-0022
  14. Anastasi et al., 2013)Anastasi, G., Antonelli, M., Bechini, A., Brienza, S., D'Andrea, E., De Guglielmo, D., … Segatori, A. (2013). Urban and social sensing for sustainable mobility in smart cities. 2013 Sustainable Internet and ICT for Sustainability, SustainIT 2013. https://doi.org/10.1109/SustainIT.2013.6685198
  15. Mikolov, Chen, Corrado, & Dean, 2013)Mikolov, T., Chen, K., Corrado, G., & Dean, J. (2013). Efficient Estimation of Word Representations in Vector Space. 1-12. Retrieved from http://arxiv.org/abs/1301.3781
  16. Baroni, Dinuand, & Kruszewski, 2014)Baroni, M., Dinuand, G., & Kruszewski, G. (2014). Don't count, predict! A systematic comparison of context-counting vs. context-predicting semantic vectors. Proceedings for the 52nd Annual Meeting for the Association of Computational Linguists, 238-247.
  17. Konstas, 2014)Konstas, I. (2014). Joint Models for Concept-to-text Generation.
  18. E., P., B., & F., 2015)E., D., P., D., B., L., & F., M. (2015). Real-Time Detection of Traffic from Twitter Stream Analysis. IEEE Transactions on Intelligent Transportation Systems, 16(4), 2269-2283. https://doi.org/10.1109/TITS.2015.2404431
  19. Farzindar & Wael, 2015)Farzindar, A., & Wael, K. (2015). a Survey of Techniques for Event Detection in Twitter. Computational Intelligence, 31(1), 132-164.
  20. Gutierrez, Figuerias, Oliveira, Costa, & Jardim-Goncalves, 2015)Gutierrez, C., Figuerias, P., Oliveira, P., Costa, R., & Jardim-Goncalves, R. (2015). Twitter mining for traffic events detection. Proceedings of the 2015 Science and Information Conference, SAI 2015, 371-378. https://doi.org/10.1109/SAI.2015.7237170
  21. Sidhaye & Cheung, 2015)Sidhaye, P., & Cheung, J. C. K. (2015). Indicative Tweet Generation: An Extractive Summarization Problem? (September), 138-147. https://doi.org/10.18653/v1/d15- 1014
  22. Campello, Moulavi, Zimek, & Sander, 2015)Campello, R. J. G. B., Moulavi, D., Zimek, A., & Sander, J. (2015). Hierarchical Density Estimates for Data Clustering, Visualization, and Outlier Detection. ACM Transactions on Knowledge Discovery from Data, 10(1), 1-51. https://doi.org/10.1145/2733381
  23. Medina & Ramon, 2015)Medina, C. P., & Ramon, M. R. R. (2015). Using TF-IDF to Determine Word Relevance in Document Queries Juan. New Educational Review, 42(4), 40-51. https://doi.org/10.15804/tner.2015.42.4.03
  24. Tran & Popowich, 2016)Tran, K., & Popowich, F. (2016). Automatic Tweet Generation From Traffic Incident Data. 59-66. https://doi.org/10.18653/v1/w16-3512
  25. McInnes & Healy, 2017)McInnes, L., & Healy, J. (2017). Accelerated Hierarchical Density Based Clustering. IEEE International Conference on Data Mining Workshops, ICDMW, 2017- Novem, 33-42. https://doi.org/10.1109/ICDMW.2017.12
  26. McInnes, Healy, & Astels, 2017)McInnes, L., Healy, J., & Astels, S. (2017). hdbscan: Hierarchical density based clustering. The Journal of Open Source Software, 2(11), 205. https://doi.org/10.21105/joss.00205
  27. Sherstinsky, 2018)Sherstinsky, A. (2018). Fundamentals of Recurrent Neural Network (RNN) and Long Short-Term Memory (LSTM) Network. 1-39. Retrieved from http://arxiv.org/abs/1808.03314
  28. Natural Language Toolkit NLTK 3.4.3 documentation, https://www.nltk.org/, last accessed 16/05/2019
  29. pandas Python Data Analysis Library, https://pandas.pydata.org/, last accessed 15/05/2019
  30. Industrial-Strength Natural Language Processing in Python, https://spacy.io/, last accessed 16/05/2019
  31. gensim topic modelling for humans, https://radimrehurek.com/gensim/, last accessed 17/05/2019
  32. scikit-learn Machine learning in Python, https://scikit-learn.org/stable/, last accessed 27/05/2019
  33. NumPy, https://www.numpy.org/, last accessed 27/05/2019
  34. matplotlib, https://matplotlib.org/, last accessed 16/05/2019
  35. BokehJS, https://bokeh.pydata.org/en/latest/docs/dev_guide/bokehjs.html, last accessed 16/05/2019
  36. The hdbscan Clustering Library, https://hdbscan.readthedocs.io/en/latest/, last accessed 20/05/2019
  37. HDBSCAN, Fast Density based Clustering, the How and the Why -John Healy, https://www.youtube.com/watch?v=dGsxd67IFiU&feature=youtu.be, last accessed 20/05/2019
  38. tweet-clustering, https://github.com/pksohn/tweet-clustering, last accessed 20/05/2019 38. Comparing Python Clustering Algorithms, https://hdbscan.readthedocs.io/en/latest/comparing_clustering_algorithms.html, last ac- cessed 20/05/2019
  39. The Illustrated Word2vec, https://jalammar.github.io/illustrated-word2vec/, last accessed 17/05/2019
  40. Do More with Twitter Data, https://twitterdev.github.io/do_more_with_twitter_data/clustering-users.html, last accessed 14/05/2019
  41. plt.ylabel('# of tweets') plt.xlabel('cluster label') plt.title('Classification of tweets with kmeans');

Related papers

Security in Computer and Information Sciences
Tadeusz Czachorski

Springer eBooks, 2018

The Internet of Things (IoT) was born in the mid 2010's, when the threshold of connecting more objects than people to the Internet, was crossed. Thus, attacks and threats on the content and quality of service of the IoT platforms can have economic, energetic and physical security consequences that go way beyond the traditional Internet's lack of security, and way beyond the threats posed by attacks to mobile telephony. Thus, this paper describes the H2020 project "Secure and Safe Internet of Things" (SerIoT) which will optimize the information security in IoT platforms and networks in a holistic, cross-layered manner (i.e. IoT platforms and devices, honeypots, SDN routers and operator's controller) in order to offer a secure SerIoT platform that can be used to implement secure IoT platforms and networks anywhere and everywhere.

View PDFchevron_right
Editorial: Special issue on security and privacy in smart cyber-physical systems
Doug Jacobson

Computers & Security, 2020

Recent years have witnessed the rapid development and deployment of the Internet of Things (IoT) thanks to the tremendous advancement of computer and communication technologies [4, 7]. Also, the proliferation of IoT devices in the past years, ranging from body sensors and wearable devices to home appliances and industrial monitoring sensors, has driven the rapid evolution of various computing paradigms such as pervasive and ubiquitous computing, mobile crowd sensing, edge computing, as well as the emergence of diverse applications and services like smart cities, eHealthcare, intelligent transportation systems [2, 6]. The resulting networking and computing paradigms, applications and services together have expedited the rising of the big data era, and significantly changed our daily life. For instance, the audio intelligent system of Amazon, Echo, can learn the habits of the customers so as to customise the unique audio services. Many other intelligent systems continue appearing [7]. Despite the technological and societal advantages and huge economical potentials of IoT, it is widely recognised that security and piracy has become one of the major concerns, which seriously impede the further development and deployment of IoT infrastructures, services, and applications [7]. Due to the intrinsic complexity and the huge amount of stand-alone devices involved in the IoT, the asymmetric advantages between Cyber defenders and attackers would World Wide Web

View PDFchevron_right
Proceedings of the WESS'15: Workshop on Embedded Systems Security - WESS'15
Stavros Koubias

2015

View PDFchevron_right
Information Security Theory and Practice. Smart Devices, Pervasive Systems, and Ubiquitous Networks, Third IFIP WG 11.2 International Workshop, WISTP 2009, Brussels, Belgium, September 1-4, 2009, Proceedings
Olivier Markowitch

Lecture Notes in Computer Science, 2009

View PDFchevron_right
Information Security Theory and Practice. Smart Devices, Pervasive Systems, and Ubiquitous Networks
Olivier Markowitch

Springer eBooks, 2009

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved