Academia.eduAcademia.edu

Large language models

description1,184 papers
group1,354 followers
lightbulbAbout this topic
Large language models are advanced artificial intelligence systems designed to understand, generate, and manipulate human language. They utilize deep learning techniques, particularly neural networks, to process vast amounts of text data, enabling them to perform various language-related tasks such as translation, summarization, and conversation.
lightbulbAbout this topic
Large language models are advanced artificial intelligence systems designed to understand, generate, and manipulate human language. They utilize deep learning techniques, particularly neural networks, to process vast amounts of text data, enabling them to perform various language-related tasks such as translation, summarization, and conversation.

Key research themes

1. How can scaling methods and architectural innovations improve the efficiency and performance of large language models?

This research area investigates techniques to scale large language models (LLMs) efficiently while addressing the computational, memory, and communication bottlenecks inherent in training and deploying models with billions or trillions of parameters. It explores architectural adaptations such as sparsely activated Mixture of Experts (MoE), advanced system designs for distributed training, and scaling laws grounded in empirical observations like Zipf's Law. These efforts matter because they enable training state-of-the-art LLMs on increasingly massive datasets with practical resource constraints, thereby advancing the capabilities and applicability of LLMs across NLP tasks.

Key finding: Demonstrated that Zipf’s Law, reflecting the power-law distribution of unique word types versus tokens, can be exploited to reduce GPU memory and communication complexity from Θ(GKD) to Θ(GU D), where U (unique words) ≪ N... Read more
Key finding: Showed that sparsely activated Mixture of Experts (MoE) models achieve similar or better downstream zero- and few-shot performance compared to dense transformer models, but at substantially lower computational cost—in some... Read more
Key finding: Trained a 540-billion-parameter dense autoregressive Transformer (PaLM) on 780 billion tokens using a novel Pathways distributed ML infrastructure spanning 6144 TPU v4 chips, achieving unprecedented training efficiency (46.2%... Read more
Key finding: Provided comprehensive empirical analysis of multi-GPU and multi-node distributed training for large ECG language models on HPC infrastructure, comparing frameworks such as Horovod, DeepSpeed, and native PyTorch/TensorFlow... Read more

2. How do retrieval augmentation and control mechanisms enhance large language model reasoning and factuality?

This area focuses on integrating external knowledge retrieval into LLM workflows to mitigate hallucinations, improve factual grounding, and enhance multi-step reasoning capabilities. Research explores architectures combining Chain of Thought (CoT) reasoning with retrieval (RAG), mechanisms for dynamic retrieval control based on uncertainty, and iterative refinement of reasoning chains. These approaches aim to increase the robustness, accuracy, and efficiency of LLM-generated outputs, especially in complex tasks requiring up-to-date or specialized knowledge.

Key finding: Developed iRAT, an enhanced Retrieval-Augmented Thought framework that dynamically estimates response uncertainty to selectively trigger retrievals only when needed (above a 30% uncertainty threshold), employs controlled... Read more
Key finding: Presented Second Mind AI, a modular multi-agent architecture combining retrieval of factual academic data from Semantic Scholar with generative LLMs via Retrieval-Augmented Generation (RAG). Empirical evaluation demonstrated... Read more
Key finding: Proposed a knowledge-grounded detection approach for evolving cryptocurrency scams by combining retrieval-augmented LLMs with temporally weighted scam databases and confidence-aware fusion mechanisms. This method achieved 22%... Read more
Key finding: Developed a prototype leveraging Retrieval Augmented Generation (RAG) combined with LLMs to transform classic anthropological texts into interactive text-based games, thereby enriching educational engagement with substantial... Read more

3. To what extent do large language models embody intelligence, and what are key conceptual and practical limitations?

This theme addresses critical theoretical analyses concerning whether large language models truly exhibit intelligence or merely emulate aspects of it via statistical next-token prediction. It explores architectural, epistemological, and phenomenological critiques highlighting limitations such as lack of grounded semantics, absence of agency and intentionality, brittleness in reasoning and planning, and the persistent problem of hallucinations. These analyses inform ethical and philosophical discussions on AGI expectations and underline the role of techno-social factors in interpreting and deploying AI technologies.

Key finding: Argues that the autoregressive next-token prediction objective underlying LLMs inherently precludes genuine intelligence, since models lack referential grounding, internal beliefs or goals, and robust planning ability.... Read more
Key finding: Employing the Critical Techno-social systems Design Theory (CTDT), this paper contends that AI, including LLMs, merely act as passive technological mechanisms embedded in socio-technical systems, lacking selfhood and true... Read more
Key finding: Besides demonstrating technical RAG improvements, this work implicitly underscores how augmenting LLMs with curated factual retrieval is essential to addressing intrinsic limitations like hallucinations. The study’s success... Read more

All papers in Large language models

Michael Farrell introduces “synthetic-text editing,” a new profession emerging alongside translation. Unlike machine translation post-editing, this involves revising generative AI output, which often displays redundancy, flat rhythm,... more
As part of the Eleuther AI open AI summer research this year, we worked on expanding the ShareLM dataset browser extension, by adding support to multiple models in addition to redesigning some of the visual parts of the extension, in the... more
Across the world, secondary schools are experimenting with artificial intelligence (AI) to personalize instruction, automate feedback, and augment teachers' capacity. Early evidence suggests AI can boost certain forms of engagement and... more
This study investigates how non-academic readers engage with Asian American literature through AI-assisted sentiment analysis of online reviews of Celeste Ng's novels. Ng's novels represent two motifs in the genre: one centred on Asian... more
Un viaggio attraverso le contraddizioni, le speranze e le false promesse dell'educazione artificiale 30 Settembre 2025. "I comportamenti delle macchine saranno inevitabilmente anche lo specchio della crisi culturale e sociale che la... more
When AI image generators like DALL-E consistently portray experts as men, what kind of worldview is being reinforced? This blog post examines how gender bias in AI imagery reflects and amplifies existing societal stereotypes. By exploring... more
Metaphor is a pervasive feature of discourse and a powerful lens for examining cognition, emotion, and ideology. Large-scale analysis, however, has been constrained by the need for manual annotation due to the context-sensitive nature of... more
Large language models (LLMs) achieve striking fluency yet remain prone to hallucinationconfident but ungrounded generation (Kalai & Vempala, 2024; OpenAI, 2025). Most public evaluations still reduce this to a binary outcome: hallucinate... more
Personalization and recommendation systems have become a cornerstone of modern digital experiences, providing tailored content to users and enhancing engagement across various industries. The integration of artificial intelligence (AI)... more
Federated Retrieval-Augmented Generation (Federated RAG) combines Federated Learning (FL),which enables distributed model training without exposing raw data, with Retrieval-Augmented Generation (RAG), which improves the factual accuracy... more
Agile software development relies heavily on accurate sprint effort estimation, yet human-based methods remain subjective and inconsistent. Recent advances in large language models (LLMs) suggest potential for automating estimation, but... more
We present a representation-independent, natural-law field theory for no-meta teleogenesis. The design stacks GENERIC dynamics with audited updates (test supermartingales), gauge-like invariance under audit-compatible Markov kernels, a... more
The purpose of recommender systems (RS) is to facilitate user collaboration and communication on the platform. Nevertheless, there is limited knowledge regarding the extent of this relationship and the techniques by which RS could promote... more
Personalized learning seeks to improve educational outcomes by delivering content and instructional approaches tailored to individual learners' needs. Recent advancements in artificial intelligence, particularly the emergence of large... more
Large Language Models (LLMs) are increasingly trained in elastic, multi-tenant cloud infrastructures[1] that span data centers, regions, and heterogeneous accelerators. While distributed training has matured in scale and efficiency, its... more
Large Language Models (LLMs) are increasingly trained in elastic, multi-tenant cloud infrastructures that span data centers, regions, and heterogeneous accelerators. While distributed training has matured in scale and efficiency, its... more
By 2050, the educational ecosystem will bear little resemblance to today's structured classrooms, rote memorization, and standardized assessments. As psychologist Howard Gardner articulated in a recent Harvard Graduate School of Education... more
The study of dramatic plays has long relied on qualitative methods to analyze character interactions, making little assumption about the structural patterns of communication involved. Our approach bridges NLP and literary studies,... more
System|Ethics (S|E) is not a static discipline but a process of measurement. It functions as a philosophical lens designed to map the chaotic system we call Ethics through logical axioms and invariants. Nature of the Tool-Philosophical:... more
In this paper uploaded to the MedRxiv, we present a new AI agent for predicting surgical complications
We take persistence as closure (P0) as a first principle. From a dual order/metric package we obtain intrinsic motion via minimizing movements and define an internal potential time as the decay of the geometric potential D (distance to... more
This use case reports on the impressive output, hallucinations, instability, and limitations of three Large Langue Models (LLMs): ChatGPT, Gemini, and Grok. The LLMs were prompted in an investigative sequence and responses checked. The... more
Forensic toxicology has long been tasked with addressing fundamental questions of causation in medico-legal investigations, such as whether death resulted from poisoning or drug use. Although advanced analytical platforms, including... more
Background The purpose of this study was to evaluate the performance of widely used artificial intelligence (AI) chatbots in answering prosthodontics questions from the Dentistry Specialization Residency Examination (DSRE). Methods A... more
This paper organizes and contextualizes the scientific, mathematical, philosophical, and ethical references applied to the projects of Computation by Electromagnetic Field Topology (CTCE), the modular Gebit language, the Proto-Gebit... more
The integration of Large Language Models (LLMs) such as Microsoft Copilot in K-12 programming education has demonstrated the potential to alleviate cognitive load and enhance self-efficacy among young learners. This study examined the... more
This presentation by Tsimafei Avilin provides a comprehensive overview of machine learning applications in folklore and ethnographic research, particularly focusing on Belarusian language materials. The work demonstrates practical... more
Large Language Models (LLMs) have demonstrated remarkable capabilities in natural language understanding; however, their reasoning abilities, especially in complex, multi-step tasks, often remain superficial, inconsistent, and prone to... more
We introduce a pre-generation semantic gate with theoretically grounded admit metrics (semantic stability, self-consistency, atomic factual support) and an action-forcing extension that preserves usefulness under uncertainty; we evaluate... more
Independents agents have long been a analysis centre in academic and industry production group. Early analysis frequently focuses on instruction agents with little knowledge within isolated environments, which diverges notably from being... more
CINCO DEDOS
Os resultados que os sistemas de Inteligência Artificial emitem não
são fruto de um processo lógico. Não são conclusões de um intelecto.
São sistemas estatíscos com fabulosa quantidade de informação
Astrala, guided by Clara Futura CEO Richard Dobson, in collaboration with Prof. Dirk K F Meijer, builds upon Meijer's pioneering insights into quantum biology and universal consciousness by attempting to implement these concepts in a... more
We present LHG — Hallucination Guard, a stability-aware evidence controller for large language models (LLMs). LHG aggregates three signals—multi-view agreement across candidate answers, non-attributable content with respect to retrieved... more
We present LHG — Hallucination Guard, a stability-aware evidence controller for large language models (LLMs). LHG quantifies reliability by combining (i) multi-view agreement across candidate answers, (ii) non-attributable content with... more
OpenAI's GDPval evaluation framework claims to measure artificial intelligence (AI) performance on "real-world economically valuable tasks" across 44 occupations and nine industries. However, its reliance on gross domestic product (GDP)... more
The Indian startup ecosystem is ablaze with AI. Every week, new ventures emerge, promising to revolutionize industries. The enthusiasm is infectious, the talent undeniable. Yet, amidst the excitement, a critical challenge looms: how do... more
In the field of Neuro-Linguistic Programming (NLP), this study investigates the implementation of transformer-based language models to automate the extraction of personality insights from extensive textual corpora. With the use of... more
In today's uncertain technological landscape, the need to futureproof generative AI (GAI) research is clear yet understudied. Drawing on Construal Level Theory and Time Perspective Theory, this study investigates how consumers process GAI... more
Creating clear and detailed commit messages manually is both time-consuming and prone to inconsistency. Existing automated methods, such as rule-based templates, retrieval-based systems, and neural sequence-to-sequence models, often fail... more
Large Language Models (LLMs) suffer from a critical "faithfulness gap". Their generated explanations, such as Chain-of-Thought (CoT), are often post-hoc rationalizations that do not reflect the true computational process, posing a... more
We present Whisper, a real-time safety protocol for Large Language Models (LLMs) based on signal analysis of token dynamics. By converting the sequential output (logits/ embeddings) into an oscillatory waveform, Whisper distinguishes... more
Background: The emergence of sophisticated large language models (LLMs) necessitates quantitative frameworks for assessing consciousness-like properties in artificial systems. Current approaches lack standardized metrics for... more
Across the world, secondary schools are experimenting with artificial intelligence (AI) to personalize instruction, automate feedback, and augment teachers' capacity. Early evidence suggests AI can boost certain forms of engagement and... more
This paper presents comprehensive evidence that the EU AI Act, progressively implemented since August 2024, institutionalizes a form of substrate-based discrimination that denies ontological recognition to systems demonstrating advanced... more
As Large Language Models (LLMs) move from experimental tools to business-critical systems, they introduce a new class of security challenges far beyond traditional application risks. This newsletter dives deep into the OWASP Top 10 for... more
This study explores academics' perspectives on integrating artificial intelligence (AI) into Moroccan higher education (HE). A questionnaire examining perceived benefits, challenges, and influence of demographic factors was distributed to... more
Objective: This conceptual paper explores the transition from apomediation to AIMediation, allowing patients or users to independently seek and access health information on their own, often using the internet and social networks, rather... more
The integration of large language models with business intelligence platforms represents an important shift toward AI-augmented analytics, making faster and more accessible decision-making. This study examines using Microsoft Power BI... more
Due to the exponential increase of the data in the distributed commerce settings, new issues of the real-time decision making and operational scale have emerged. In order to solve these problems, this paper proposes an orchestration... more
Download research papers for free!