Rinat Sergeev

Harvard University, IQSS, Faculty Member

Followers

Following

Co-authors

Public Views

Address: Boston, Massachusetts, United States

less

Interests

Uploads

Papers by Rinat Sergeev

Relative Feedback Increases Disparities in Effort and Performance in Crowdsourcing Contests

Proceedings of the ACM on Human-Computer Interaction

Rankings and leaderboards are often used in crowdsourcing contests and online communities to moti... more Rankings and leaderboards are often used in crowdsourcing contests and online communities to motivate individual contributions but feedback based on social comparison can also have negative effects. Here, we study the unequal effects of such feedback on individual effort and performance for individuals of different ability. We hypothesize that the effects of social comparison differ for top performers and bottom performers in a way that the inequality between the two increases. We use a quasi-experimental design to test our predictions with data from Topcoder, a large online crowdsourcing platform that publishes computer programming contests. We find that in contests where the submitted code is evaluated against others' submissions, rather than using an absolute scale, top performers increase their effort while bottom performers decrease it. As a result, relative scoring leads to better outcomes for those at the top but lower engagement for bottom performers. Our findings expose...

Download

Only Time Will Tell!

PsycCRITIQUES, 1974

PDS and NASA Tournament Laboratory Engaging Developers: The Rings Challenge

Interpreting the effect of vaccination on steady state infection in animals challenged with Simian immunodeficiency virus

by Igor Rouzine, John Coffin, and Rinat Sergeev

Journal of Theoretical Biology, 2010

A representative vaccinated macaque challenged with SIVmac251 establishes a persistent infection ... more A representative vaccinated macaque challenged with SIVmac251 establishes a persistent infection with a lower virus load, higher CTL frequencies, and much higher helper cell frequencies, than a representative control animal. The reasons for the difference are not fully understood. Here we interpret this effect using a mathematical model we developed recently to explain results of various experiments on virus and CTL dynamics in SIV-infected macaques and HIV-infected humans. The model includes two types of cytotoxic lymphocytes (CTLs) regulated by antigen-activated helper cells and directly by infected cells, respectively, and predicts the existence of two steady states with different viremia, helper cell and CTL levels. Depending on the initial level of CTL memory cells and helper cells, a representative animal ends up in either the high-virus state or the low-virus state, which accounts for the observed differences between the two animal groups. Viremia in the low-virus state is proportional to the antigen sensitivity threshold of helper cells. Estimating the infectivity ratio of activated and resting CD4 T cells at 200-300, the correct range for the critical memory cell percentage and the viremia peak suppression is predicted. However, the model does not explain why viremia in the ''low-virus state'' is surprisingly high , relative to vaccinated animals infected with SHIV, and broadly distributed among challenged animals. We conclude that the model needs an update explaining extremely low sensitivity of uninfected helper cells to antigen in vaccinated animals.

Download

Detections of Propellers in Saturn's Rings using Machine Learning: Preliminary Results

An empirical investigation of adaptive search in problem solving

Rinat Sergeev Drug-Resistant Tuberculosis Modeling the Dynamic Relationship Between HIV and the Risk of

that longitudinal studies, not a snapshot, will provide a better picture of the whole evolving st... more that longitudinal studies, not a snapshot, will provide a better picture of the whole evolving story. . suggest et al perhaps as a result of less stringent selective pressure. Thus, Sergeev −− HIV-seronegative populations drug-resistant TB in HIV-seropositive populations may actually be less fit than drug-resistant TB that develops in rare. Intriguingly, although HIV infection may increase the prevalence of drug-resistant TB within a population, infections, acquired at a time when drug-resistant TB was M. Tb explained by HIV-stimulated reactivation of latent developing drug-resistant TB at the early stages of the coepidemic. Although counterintuitive, these results may be infections within a community over several decades, HIV-infected individuals may actually be at lower relative risk of drug-resistant TB. They found that, whereas HIV infection facilitated the rise in numbers of drug-resistant TB The authors developed a mathematical model to explore the effect of HIV on the dynamic...

Improving deconvolution methods in biology through open innovation competitions: an application to the connectivity map

Bioinformatics

Motivation Do machine learning methods improve standard deconvolution techniques for gene express... more Motivation Do machine learning methods improve standard deconvolution techniques for gene expression data? This article uses a unique new dataset combined with an open innovation competition to evaluate a wide range of approaches developed by 294 competitors from 20 countries. The competition’s objective was to address a deconvolution problem critical to analyzing genetic perturbations from the Connectivity Map. The issue consists of separating gene expression of individual genes from raw measurements obtained from gene pairs. We evaluated the outcomes using ground-truth data (direct measurements for single genes) obtained from the same samples. Results We find that the top-ranked algorithm, based on random forest regression, beat the other methods in accuracy and reproducibility; more traditional gaussian-mixture methods performed well and tended to be faster, and the best deep learning approach yielded outcomes slightly inferior to the above methods. We anticipate researchers in t...

Download

Social comparison increases the dispersion and predictability of effort and performance

Ranking in contests and competitions is often used to motivate individual contributions but it ca... more Ranking in contests and competitions is often used to motivate individual contributions but it can have negative impact on the group. Here, we study the effect of social comparison on the dispersion and predictability of individual performance. We hypothesize that the effects of social comparison on effort will differ for top-performers and bottom-performers in a way that the inequality between the two increases. We use a quasi-experimental design to test our predictions with data from Topcoder, an online crowdsourcing platform that publishes computer programming contests. We find that in contests where the submitted code is evaluated against others' submissions, rather than using an absolute scale, top-performers increase their effort while bottom-performers decrease it. As a result, relative scoring leads to a higher dispersion and somewhat higher predictability of performance, strengthening the status quo and precluding under-performers from engaging and learning. Our finding...

Innovation Contests for High-Tech Procurement

Research-Technology Management

Download

Improving Deconvolution Methods in Biology through Open Innovation Competitions: An Application to the Connectivity Map

A recurring problem in biomedical research is how to isolate signals of distinct populations (cel... more A recurring problem in biomedical research is how to isolate signals of distinct populations (cell types, tissues, and genes) from composite measures obtained by a single analyte or sensor. Existing computational deconvolution approaches work well in many specific settings, but they might be suboptimal in more general applications. Here, we describe new methods that were obtained via an open innovation competition. The goal of the competition was to characterize the expression of 1,000 genes from 500 composite measurements, which constitutes the approach of a new assay, called L1000, used to scale-up the Connectivity Map (CMap) — a catalog of millions of perturbational gene expression profiles. The competition used a novel dataset of 2,200 profiles and attracted 294 competitors from 20 countries. The top-nine performing methods ranged from machine learning approaches (Convolutional Neural Networks and Random Forests) to more traditional ones (Gaussian Mixtures and k-means). These so...

Download

Advancing computational biology and bioinformatics research through open innovation competitions

PLOS ONE

Open data science and algorithm development competitions offer a unique avenue for rapid discover... more Open data science and algorithm development competitions offer a unique avenue for rapid discovery of better computational strategies. We highlight three examples in computational biology and bioinformatics research in which the use of competitions has yielded significant performance gains over established algorithms. These include algorithms for antibody clustering, imputing gene expression data, and querying the Connectivity Map (CMap). Performance gains are evaluated quantitatively using realistic, albeit sanitized, data sets. The solutions produced through these competitions are then examined with respect to their utility and the prospects for implementation in the field. We present the decision process and competition design considerations that lead to these successful outcomes as a model for researchers who want to use competitions and non-domain crowds as collaborators to further their research.

Download

Use of Crowd Innovation to Develop an Artificial Intelligence–Based Solution for Radiation Therapy Targeting

JAMA Oncology

IMPORTANCE Radiation therapy (RT) is a critical cancer treatment, but the existing radiation onco... more IMPORTANCE Radiation therapy (RT) is a critical cancer treatment, but the existing radiation oncologist work force does not meet growing global demand. One key physician task in RT planning involves tumor segmentation for targeting, which requires substantial training and is subject to significant interobserver variation. OBJECTIVE To determine whether crowd innovation could be used to rapidly produce artificial intelligence (AI) solutions that replicate the accuracy of an expert radiation oncologist in segmenting lung tumors for RT targeting. DESIGN, SETTING, AND PARTICIPANTS We conducted a 10-week, prize-based, online, 3-phase challenge (prizes totaled $55 000). A well-curated data set, including computed tomographic (CT) scans and lung tumor segmentations generated by an expert for clinical care, was used for the contest (CT scans from 461 patients; median 157 images per scan; 77 942 images in total; 8144 images with tumor present). Contestants were provided a training set of 229 CT scans with accompanying expert contours to develop their algorithms and given feedback on their performance throughout the contest, including from the expert clinician. MAIN OUTCOMES AND MEASURES The AI algorithms generated by contestants were automatically scored on an independent data set that was withheld from contestants, and performance ranked using quantitative metrics that evaluated overlap of each algorithm's automated segmentations with the expert's segmentations. Performance was further benchmarked against human expert interobserver and intraobserver variation. RESULTS A total of 564 contestants from 62 countries registered for this challenge, and 34 (6%) submitted algorithms. The automated segmentations produced by the top 5 AI algorithms, when combined using an ensemble model, had an accuracy (Dice coefficient = 0.79) that was within the benchmark of mean interobserver variation measured between 6 human experts. For phase 1, the top 7 algorithms had average custom segmentation scores (S scores) on the holdout data set ranging from 0.15 to 0.38, and suboptimal performance using relative measures of error. The average S scores for phase 2 increased to 0.53 to 0.57, with a similar improvement in other performance metrics. In phase 3, performance of the top algorithm increased by an additional 9%. Combining the top 5 algorithms from phase 2 and phase 3 using an ensemble model, yielded an additional 9% to 12% improvement in performance with a final S score reaching 0.68. CONCLUSIONS AND RELEVANCE A combined crowd innovation and AI approach rapidly produced automated algorithms that replicated the skills of a highly trained physician for a critical task in radiation therapy. These AI algorithms could improve cancer care globally by transferring the skills of expert clinicians to under-resourced health care settings.

Download

Advancing Computational Biology and Bioinformatics Research Through Open Innovation Competitions

Open data science and algorithm development competitions offer a unique avenue for rapid discover... more Open data science and algorithm development competitions offer a unique avenue for rapid discovery of better computational strategies. We highlight three examples in computational biology and bioinformatics research where the use of competitions has yielded significant performance gains over established algorithms. These include algorithms for antibody clustering, imputing gene expression data, and querying the Connectivity Map (CMap). Performance gains are evaluated quantitatively using realistic, albeit sanitized, data sets. The solutions produced through these competitions are then examined with respect to their utility and the prospects for implementation in the field. We present the decision process and competition design considerations that lead to these successful outcomes as a model for researchers who want to use competitions and non-domain crowds as collaborators to further their research.

Download

Report on the Cloud-Based Evaluation Approaches Workshop 2015

ACM SIGIR Forum, 2016

Data analysis requires new approaches in many domains for evaluating tools and techniques, partic... more Data analysis requires new approaches in many domains for evaluating tools and techniques, particularly when the data sets grow large and more complex. Evaluation-as-aservice (EaaS) was coined as a term to represent evaluation approaches based on APIs, virtual machines or source code submission, different from the common paradigm of evaluating techniques on a distributed test collection, tasks and submitted results files. Such new approaches become necessary when data sets become extremely large, contain confidential information or might change quickly over time. The workshop on cloud-based evaluation (CBE) took place in Boston, MA, USA on November 5, 2015 and explored several approaches for data analysis evaluation and frameworks in this field. The objective was to include several stakeholders from academic partners, companies to funding agencies to cover various interests and viewpoints in the discussion of evaluation infrastructures. The workshop focused on the biomedical domain but the results are easily applicable to many domains of information analysis and retrieval.

Download

Singlet and Triplet States of X+ and X- Trions in a 2D Quantum Well

Ground-state energy of X− and X+ trions in a two-dimensional quantum well at an arbitrary mass ratio

Physics of the Solid State, 2001

ABSTRACT A simple model variational function is proposed for an adequate unified description of X... more

The Heavy-Hole X+ Trion in Double Quantum Wells

Optical Properties of 2D Systems with Interacting Electrons, 2003

ABSTRACT The system of two heavy holes in 2D Quantum Well bound by an electron in the adjacent 2D... more ABSTRACT The system of two heavy holes in 2D Quantum Well bound by an electron in the adjacent 2D Quantum Well is considered. Simple variational wave function with a few variational parameters is suggested to calculate the ground state energy of such X+ trion in the whole range of the distances between the wells. The trion appears to be bound up to unexpectedly large distance values. The critical value of the latter is found to be more than 34 (in atomic units) which is 68 times of the distance between the holes in 2D X+ trion. The resonant state of the trion at the distances between QWs greater than the critical is also considered. A few simple estimations of the trion binding energy for GaAs and ZnSe double quantum well structures are proposed.

The route of HIV escape from immune response targeting multiple sites is determined by the cost-benefit tradeoff of escape mutations

PLoS computational biology, 2014

Cytotoxic T lymphocytes (CTL) are a major factor in the control of HIV replication. CTL arise in ... more Cytotoxic T lymphocytes (CTL) are a major factor in the control of HIV replication. CTL arise in acute infection, causing escape mutations to spread rapidly through the population of infected cells. As a result, the virus develops partial resistance to the immune response. The factors controlling the order of mutating epitope sites are currently unknown and would provide a valuable tool for predicting conserved epitopes. In this work, we adapt a well-established mathematical model of HIV evolution under dynamical selection pressure from multiple CTL clones to include partial impairment of CTL recognition, [Formula: see text], as well as cost to viral replication, [Formula: see text]. The process of escape is described in terms of the cost-benefit tradeoff of escape mutations and predicts a trajectory in the cost-benefit plane connecting sequentially escaped sites, which moves from high recognition loss/low fitness cost to low recognition loss/high fitness cost and has a larger slope...

Download

The simplest electron-hole complexes localized at longitudinal fluctuations in quantum wires

Semiconductors, 2009

ABSTRACT The variational method is used to study how the states of electron-hole complexes in the... more ABSTRACT The variational method is used to study how the states of electron-hole complexes in the quantum wires of a semiconductor are influenced by extra localization along the axis of the structure. Simple trial functions involving a small number of physically substantiated adjustable parameters are suggested, which provides reasonable accuracy in calculating the binding energies of ground states of the complexes in a wide range of parameters of the system. The general form of the trial functions is independent of the specific shape of the irregularity potential. The applicability of the method is demonstrated by the examples of excitons and X + and X − trions localized at an irregularity described by a parabolic potential. The error of the method developed here to describe localization of the complexes at arbitrarily shaped potential irregularities is estimated. The accuracy of the variational calculations of the binding energy of localized excitons is estimated by comparing the results with those obtained by direct diagonalization of the Hamiltonian.

Rinat Sergeev

Uploads

Papers by Rinat Sergeev

Log In