Papers by Amit Kumar Jaiswal
User implicit feedback plays an important role in recommender systems. However, finding implicit ... more User implicit feedback plays an important role in recommender systems. However, finding implicit features is a tedious task. This paper aims to identify users' preferences through implicit behavioural signals for image recommendation based on the Information Scent Model of Information Foraging Theory. In the first part, we hypothesise that the users' perception is improved with visual cues in the images as behavioural signals that provide users' information scent during information seeking. We designed a content-based image recommendation system to explore which image attributes (i.e., visual cues or bookmarks) help users find their desired image. We found that users prefer recommendations predicated by visual cues and therefore consider the visual cues as good information scent for their information seeking. In the second part, we investigated if visual cues in the images together with the images itself can be better perceived by the users than each of them on its own. ...
ArXiv, 2019
A major challenge of recommender systems is to help users locating interesting items. Personalize... more A major challenge of recommender systems is to help users locating interesting items. Personalized recommender systems have become very popular as they attempt to predetermine the needs of users and provide them with recommendations to personalize their navigation. However, few studies have addressed the question of what drives the users' attention to specific content within the collection and what influences the selection of interesting items. To this end, we employ the lens of Information Foraging Theory (IFT) to image recommendation to demonstrate how the user could utilize visual bookmarks to locate interesting images. We investigate a personalized content-based image recommendation system to understand what affects user attention by reinforcing visual attention cues based on IFT. We further find that visual bookmarks (cues) lead to a stronger scent of the recommended image collection. Our evaluation is based on the Pinterest image collection.
Pathologists find tedious to examine the status of the sentinel lymph node on a large number of p... more Pathologists find tedious to examine the status of the sentinel lymph node on a large number of pathological scans. The examination process of such lymph node which encompasses metastasized cancer cells is histopathologically organized. However, the task of finding metastatic tissues is gradual which is often challenging. In this work, we present our deep convolutional neural network based model validated on PatchCamelyon (PCam) benchmark dataset for fundamental machine learning research in histopathology diagnosis. We find that our proposed model trained with a semi-supervised learning approach by using pseudo labels on PCam-level significantly leads to better performances to strong CNN baseline on the AUC metric.
This report presents the outputs of a week-long collaboration between the Alan Turing Institute (... more This report presents the outputs of a week-long collaboration between the Alan Turing Institute (A.T.I.) and Woolfson laboratory in the School of Chemistry at the University of Bristol, to predict different states of a type of protein fold called coiled coils (CCs) (Woolfson (2017) 1 which are the structural motifs that consist of two or more α-helices winding around each other, from linear sequences of amino acids by machine learning methods.
Semantic Hilbert Space for Interactive Image Retrieval
Proceedings of the 2021 ACM SIGIR International Conference on Theory of Information Retrieval
The paper introduces a model for interactive image retrieval utilising the geometrical framework ... more The paper introduces a model for interactive image retrieval utilising the geometrical framework of information retrieval (IR). We tackle the problem of image retrieval based on an expressive user information need in form of a textual-visual query, where a user is attempting to find an image similar to the picture in their mind during querying. The user information need is expressed using guided visual feedback based on Information Foraging which lets the user perception embed within the model via semantic Hilbert space (SHS). This framework is based on the mathematical formalism of quantum probabilities and aims to understand the relationship between user textual and image input, where the image in the input is considered a form of visual feedback. We propose SHS, a quantum-inspired approach where the textual-visual query is regarded analogously to a physical system that allows for modelling different system states and their dynamic changes thereof based on observations (such as queries, relevance judgements). We will be able to learn the input multimodal representation and relationships between textual-image queries for retrieving images. Our experiments are conducted on the MIT States and Fashion200k datasets that demonstrate the effectiveness of finding particular images autocratically when the user inputs are semantically expressive.
The trending global pandemic of COVID-19 is the fastest ever impact which caused people worldwide... more The trending global pandemic of COVID-19 is the fastest ever impact which caused people worldwide by severe acute respiratory syndrome (SARS)-driven coronavirus. However, several countries suffer from the shortage of test kits and high false negative rate in PCR test. Enhancing the chest X-ray or CT detection rate becomes critical. The patient triage is of utmost importance and the use of machine learning can drive the diagnosis of chest X-ray or CT image by identifying COVID-19 cases. To tackle this problem, we propose COVIDPEN - a transfer learning approach on Pruned EfficientNet-based model for the detection of COVID-19 cases. The proposed model is further interpolated by post-hoc analysis for the explainability of the predictions. The effectiveness of our proposed model is demonstrated on two systematic datasets of chest radiographs and computed tomography scans. Experimental results with several baseline comparisons show that our method is on par and confers clinically explicab...
IEEE Access
Deep Learning-based Smart Healthcare is getting so much attention due to real-time applicability ... more Deep Learning-based Smart Healthcare is getting so much attention due to real-time applicability in everyone life's, and It has obtained more attention with the convergence of IoT. Diabetic eye disease is the primary cause of blindness between working aged peoples. The major populated Asian countries such as India and China presently account for millions of people and at the verge of an eruption of diabetic inhabitants. These growing number of diabetic patients posed a major challenge among trained doctors to provide medical screening and diagnosis. Our goal is to leverage the deep learning techniques to automate the detection of blind spot in an eye and identify how severe the stage may be. In this paper, we propose an optimized technique on top of recently released pre-trained EfficientNet models for blindness identification in retinal images along with a comparative analysis among various other neural network models. Our fine-tuned EfficientNet-B5 based model evaluation follows the benchmark dataset of retina images captured using fundus photography during varied imaging stages and outperforms CNN and ResNet50 models. INDEX TERMS Diabetic retinopathy, medical diagnosis, CNN, retina images, the IoT.
A Blockchain-Enabled Multi Domain Edge Computing Orchestrator
IEEE Internet of Things Magazine
Today, management and orchestration are considered prime components of the new network management... more Today, management and orchestration are considered prime components of the new network management layer. Multi-domain orchestration has helped in simplifying infrastructural operations and enables better scaling and faster deployment of network services. However, resource provisioning considering network optimization and fulfillment of multi-constraint quality of service (QoS) is still a major concern. In this article, an architecture comprising multi-domain edge orchestration (MDEO) entrusted by blockchain designed to solve the problem of multi-constraint QoS is proposed. A dynamic end-to-end (E2E) network slicing algorithm is devised to execute at the MDEO to enable multi-tenant on-demand network infrastructure provisioning isolation and security. The algorithm first calculates an optimum network slice topology and then instantiates the involved virtual network functions. Based on multi-constraint QoS, it fulfills the E2E slice request. Blockchain is deployed to ensure trustworthiness between different telecom operators, introduce transparency and to automate the fulfillment of service-level agreements through smart contracts. In addition, an experimental simulation of the system is performed and the burst and response times of the proposed framework are analyzed using different distributions. The data in both cases demonstrates a lognormal distributed behavior.
Proceedings of the 11th Forum for Information Retrieval Evaluation, Dec 12, 2019
User implicit feedback plays an important role in recommender systems. However, finding implicit ... more User implicit feedback plays an important role in recommender systems. However, finding implicit features is a tedious task. This paper aims to identify users' preferences through implicit behavioural signals for image recommendation based on the Information Scent Model of Information Foraging Theory. In the first part, we hypothesise that the users' perception is improved with visual cues in the images as behavioural signals that provide users' information scent during information seeking. We designed a content-based image recommendation system to explore which image attributes (i.e., visual cues or bookmarks) help users find their desired image. We found that users prefer recommendations predicated by visual cues and therefore consider the visual cues as good information scent for their information seeking. In the second part, we investigated if visual cues in the images together with the images itself can be better perceived by the users than each of them on its own. We evaluated the information scent artifacts in image recommendation on the Pinterest image collection and the WikiArt dataset. We find our proposed image recommendation system supports the implicit signals through Information Foraging explanation of the information scent model.
Neural Computing and Applications
In image-based medical decision-making, different modalities of medical images of a given organ o... more In image-based medical decision-making, different modalities of medical images of a given organ of a patient are captured. Each of these images will represent a modality that will render the examined organ differently, leading to different observations of a given phenomenon (such as stroke). The accurate analysis of each of these modalities promotes the detection of more appropriate medical decisions. Multimodal medical imaging is a research field that consists in the development of robust algorithms that can enable the fusion of image information acquired by different sets of modalities. In this paper, a novel multimodal medical image fusion algorithm is proposed for a wide range of medical diagnostic problems. It is based on the application of a boundary measured pulse-coupled neural network fusion strategy and an energy attribute fusion strategy in a non-subsampled shearlet transform domain. Our algorithm was validated in dataset with modalities of several diseases, namely glioma...
Evidence of power-law behavior in cognitive IoT applications
Neural Computing and Applications
The motivations induced due to the presence of scale-free characteristics of neural systems gover... more The motivations induced due to the presence of scale-free characteristics of neural systems governed by the well-known power-law distribution of neuronal activities have led to its convergence with the Internet of things (IoT) framework. The IoT is one such framework, where the self-organization of the connected devices is a momentous aspect. The devices involved in these networks inherently relate to the collection of several consolidated devices like the sensory devices, consumer appliances, wearables, and other associated applications, which facilitate a ubiquitous connectivity among the devices. This is one of the most significant prerequisites of IoT systems as several interconnected devices need to be included in the convolution for the uninterrupted execution of the services. Thus, in order to understand the scalability and the heterogeneity of these interconnected devices, the exponent of power-law plays a significant role. In this paper, an analytical framework to illustrate the ubiquitous power-law behavior of the IoT devices is derived. An emphasis regarding the mathematical insights for the characterization of the dynamic behavior of these devices is conceptualized. The observations made in this direction are illustrated through simulation results. Further, the traits of the wireless sensor networks, in context with the contemporary scale-free architecture, are discussed.
Measurement
The rich collection of annotated datasets piloted the robustness of deep learning techniques to e... more The rich collection of annotated datasets piloted the robustness of deep learning techniques to effectuate the implementation of diverse medical imaging tasks. Over 15% of deaths include children under age five are caused by pneumonia globally. In this study, we describe our deep learning based approach for the identification and localization of pneumonia in Chest X-rays (CXRs) images. Researchers usually employ CXRs for the diagnostic imaging study. Several factors such as positioning of the patient and depth of inspiration can change the appearance of the chest X-ray, complicating interpretation further. Our identification model (https://github.com/amitkumarj441/identify_pneumonia) is based on Mask-RCNN, a deep neural network which incorporates global and local features for pixel-wise segmentation. Our approach achieves robustness through critical modifications of the training process and a novel postprocessing step which merges bounding boxes from multiple models. The proposed identification model achieves better performances evaluated on chest radiograph dataset which depict potential pneumonia causes.
We propose Parsec, a web-scale State channel for the Internet of Value to exterminate the consens... more We propose Parsec, a web-scale State channel for the Internet of Value to exterminate the consensus bottleneck in Blockchain by leveraging a network of state channels which enable to robustly transfer value off-chain. It acts as an infrastructure layer developed on top of Ethereum Blockchain, as a network protocol which allows coherent routing and interlocking channel transfers for trade-off between parties. A web-scale solution for state channels is implemented to enable a layer of value transfer to the internet. Existing network protocol on State Channels include Raiden for Ethereum and Lightning Network for Bitcoin. However, we intend to leverage existing web-scale technologies used by large Internet companies such as Uber, LinkedIn or Netflix. We use Apache Kafka to scale the global payment operation to trillions of operations per day enabling near-instant, low-fee, scalable, and privacy-sustainable payments. Our architecture follows Event Sourcing pattern which solves current i...
Proceedings of the 42th European Conference of Information Retrieval (ECIR 2020), 2020
[0000−0001−8848−7041] , Haiming Liu 1[0000−0002−0390−3657] , and Ingo Frommholz 1[0000−0002−5622−... more [0000−0001−8848−7041] , Haiming Liu 1[0000−0002−0390−3657] , and Ingo Frommholz 1[0000−0002−5622−5132] Abstract. Query Auto-completion (QAC) is a prominently used feature in search engines, where user interaction with such explicit feature is facilitated by the possible automatic suggestion of queries based on a prefix typed by the user. Existing QAC models have pursued a little on user interaction and cannot capture a user's information need (IN) context. In this work, we devise a new task of QAC applied on an image for estimating patch (one of the key components of Information Foraging Theory) probabilities for query suggestion. Our work supports query completion by extending a user query prefix (one or two characters) to a complete query utilising a foraging-based probabilistic patch selection model. We present iBERT, to fine-tune the BERT (Bidirectional En-coder Representations from Transformers) model, which leverages combined textual-image queries for a solution to image QAC by computing probabilities of a large set of image patches. The reflected patch probabilities are used for selection while being agnostic to changing information need or contextual mechanisms. Experimental results show that query auto-completion using both natural language queries and images is more effective than using only language-level queries. Also, our fine-tuned iB-ERT model allows to efficiently rank patches in the image.
European Conference on Information Retrieval (ECIR), 2020
Query Auto-completion (QAC) is a prominently used feature in search engines, where user interacti... more Query Auto-completion (QAC) is a prominently used feature in search engines, where user interaction with such explicit feature is facilitated by the possible automatic suggestion of queries based on a prefix typed by the user. Existing QAC models have pursued a little on user interaction and cannot capture a user's information need (IN) context. In this work, we devise a new task of QAC applied on an image for estimating patch (one of the key components of Information Foraging Theory) probabilities for query suggestion. Our work supports query completion by extending a user query prefix (one or two characters) to a complete query utilising a foraging-based probabilistic patch selection model. We present iBERT, to fine-tune the BERT (Bidirectional En-coder Representations from Transformers) model, which leverages combined textual-image queries for a solution to image QAC by computing probabilities of a large set of image patches. The reflected patch probabilities are used for selection while being agnostic to changing information need or contextual mechanisms. Experimental results show that query auto-completion using both natural language queries and images is more effective than using only language-level queries. Also, our fine-tuned iB-ERT model allows to efficiently rank patches in the image.
CVPR Towards Causal, Explainable and Universal Medical Visual Diagnosis (MVD) Workshop, 2019
Pathologists find tedious to examine the status of the sen-tinel lymph node on a large number of ... more Pathologists find tedious to examine the status of the sen-tinel lymph node on a large number of pathological scans. The examination process of such lymph node which encompasses metastasized cancer cells is histopathologically organized. However, the task of finding metastatic tissues is gradual which is often challenging. In this work, we present our deep convolutional neural network based model validated on PatchCamelyon (PCam) benchmark dataset for fundamental machine learning research in histopathology diagnosis. We find that our proposed model trained with a semi-supervised learning approach by using pseudo labels on PCam-level significantly leads to better performances to strong CNN baseline on the AUC metric.
LWDA, 2018
In this paper, we present an extension, and an evaluation, to existing Quantum like approaches of... more In this paper, we present an extension, and an evaluation, to existing Quantum like approaches of word embedding for IR tasks that (1) improves complex features detection of word use (e.g., syntax and semantics), (2) enhances how this method extends these aforementioned uses across linguistic contexts (i.e., to model lexical ambiguity)-specifically Question Classification-, and (3) reduces computational resources needed for training and operating Quantum based neural networks, when confronted with existing models. This approach could also be latter applicable to significantly enhance the state-of the-art across Natural Language Processing (NLP) word-level tasks such as entity recognition, part-of-speech tagging, or sentence-level ones such as textual relatedness and entailment, to name a few.
This research concerns with translating natural language into SQL queries by exploiting the Perl ... more This research concerns with translating natural language into SQL queries by exploiting the Perl DBI library for both database construction and thesis verification in the task of question answering. We built SQGNL which uses linguistic dependencies and metadata to build sets of possible SELECT and WHERE clauses and is designed to be database and platform independent with multiuser support. It can be used by users with no knowledge of SQL to translate natural language to SQL queries. The program has the ability to learn new grammar. SQGNL application is written in Perl language with a simple user interface implemented using Tk module. It uses Perl recursive descent module to build the underlying parser. Our algorithm can be recursively applied to deal with complex questions, requiring nested SELECT instructions. Our preliminary results are encouraging as they show that our system generates the right SQL query among the first five in the 80% of the cases. This result can be greatly improved by re-ranking the queries with a machine learning algorithms.
Uploads
Papers by Amit Kumar Jaiswal