Language Models

Large Language Models (LLMs) are increasingly trained in elastic, multi-tenant cloud infrastructures[1] that span data centers, regions, and heterogeneous accelerators. While distributed training has matured in scale and efficiency, its security posture lags behind adversarial realities: training corpora may contain sensitive or regulated data; gradient channels can leak membership and attribute information; supplychain subversion can inject malicious kernels or compromised containers; cross-tenant resource sharing elevates the risk of side-channel inference; and orchestration layers, which schedule, checkpoint, and autoscale jobs[2], are attractive targets for data exfiltration and model theft. Existing mitigations remain fragmented: transport encryption protects links, confidentialcompute enclaves protect limited code paths, differential privacy protects outputs in isolation, and secure aggregation schemes address narrow communication steps. What is missing is a coherent end-to-end framework that composes these mechanisms with provable guarantees while preserving the throughput and tail-latency characteristics required by trillion-parameter training. This paper presents AegisTrain, a secure distributed training framework that treats privacy and integrity as firstclass control objectives across the full training lifecycle[3]. AegisTrain couples remotely attested confidential runtimes with attribute-bound key management to ensure that only measured and policy-compliant components can access plaintext data, gradients, and optimizer state. It introduces a cryptographic aggregation substrate that masks worker updates end-to-end and injects calibrated noise under a formally verifiable privacy accountant, rendering gradient channels useless for membership inference even when a bounded number of participants are compromised. Checkpoint images, telemetry, and intermediate artifacts are encrypted at rest with per-epoch keys derived from a hardware-rooted key hierarchy, and the supply chain is hardened with verifiable provenance, reproducible builds, and incluster policy enforcement. The framework is designed for tensor, pipeline, and data-parallel hybrids, and preserves scalability through streaming decryption, batched attestation, and offloaded cryptography that runs on CPU sidecars while GPU compute saturates the model step[4]. We develop a control-plane that enforces risk-adaptive policies-for example, tightening privacy budgets or refusing mixed-trust colocation under elevated threat intel-without manual intervention, and we present machinecheckable invariants that forbid unsafe downgrades. A queueingtheoretic and information-theoretic analysis quantifies the overhead of encryption, attestation, and privacy noise relative to communication and compute, and shows parameter regimes in which security can be achieved with sub-5% throughput loss at cluster scale. A prototype implementation with masked allreduce, enclave-gated data loaders, and encrypted checkpoints demonstrates feasibility on realistic LLM training traces. By articulating the interfaces among attestation, secure aggregation, privacy accounting, and distributed parallelism, AegisTrain reframes secure LLM training as a problem of principled composition rather than ad hoc patchwork, yielding a deployable blueprint for cloud environments where both speed and trust are non-negotiable.

Large Language Models (LLMs) are increasingly trained in elastic, multi-tenant cloud infrastructures that span data centers, regions, and heterogeneous accelerators. While distributed training has matured in scale and efficiency, its security posture lags behind adversarial realities: training corpora may contain sensitive or regulated data; gradient channels can leak membership and attribute information; supplychain subversion can inject malicious kernels or compromised containers; cross-tenant resource sharing elevates the risk of side-channel inference; and orchestration layers, which schedule, checkpoint, and autoscale jobs , are attractive targets for data exfiltration and model theft. Existing mitigations remain fragmented: transport encryption protects links, confidentialcompute enclaves protect limited code paths, differential privacy protects outputs in isolation, and secure aggregation schemes address narrow communication steps. What is missing is a coherent end-to-end framework that composes these mechanisms with provable guarantees while preserving the throughput and tail-latency characteristics required by trillion-parameter training. This paper presents AegisTrain, a secure distributed training framework that treats privacy and integrity as firstclass control objectives across the full training lifecycle . AegisTrain couples remotely attested confidential runtimes with attribute-bound key management to ensure that only measured and policy-compliant components can access plaintext data, gradients, and optimizer state. It introduces a cryptographic aggregation substrate that masks worker updates end-to-end and injects calibrated noise under a formally verifiable privacy accountant, rendering gradient channels useless for membership inference even when a bounded number of participants are compromised. Checkpoint images, telemetry, and intermediate artifacts are encrypted at rest with per-epoch keys derived from a hardware-rooted key hierarchy, and the supply chain is hardened with verifiable provenance, reproducible builds, and incluster policy enforcement. The framework is designed for tensor, pipeline, and data-parallel hybrids, and preserves scalability through streaming decryption, batched attestation, and offloaded cryptography that runs on CPU sidecars while GPU compute saturates the model step . We develop a control-plane that enforces risk-adaptive policies-for example, tightening privacy budgets or refusing mixed-trust colocation under elevated threat intel-without manual intervention, and we present machinecheckable invariants that forbid unsafe downgrades. A queueingtheoretic and information-theoretic analysis quantifies the overhead of encryption, attestation, and privacy noise relative to communication and compute, and shows parameter regimes in which security can be achieved with sub-5% throughput loss at cluster scale. A prototype implementation with masked allreduce, enclave-gated data loaders, and encrypted checkpoints demonstrates feasibility on realistic LLM training traces. By articulating the interfaces among attestation, secure aggregation, privacy accounting, and distributed parallelism, AegisTrain reframes secure LLM training as a problem of principled composition rather than ad hoc patchwork, yielding a deployable blueprint for cloud environments where both speed and trust are non-negotiable.

Building language models (LMs), especially small and medium ones, remains more art than science. While large LMs often improve by sheer scale, it is still unclear why many design choices work. For small LMs, this uncertainty is more limiting: tight parameter budgets make each decision critical, yet researchers still lack systematic, scientific ways to test and refine new ideas. We introduce Pico, a lightweight, modular framework that enables systematic, hypothesis-driven research for small and medium-scale language model development. Pico consists of two libraries that together provide a practical sandbox where researchers can make targeted changes to a model's architecture or training procedures and directly observe their effects on the model's behavior. To support reproducible experimentation, we also release a suite of baseline models, pico-decoder, trained under standardized conditions and open-sourced for the community. Case studies highlight how Pico can support iterative small LM design and analysis. pico-lm (Apache 2.0) pico-lm (Apache 2.0) picolm.io Demo Video

Recently, adversarial input highly negotiates the security concerns in deep learning (DL) techniques. The main motive to enhance the natural language processing (NLP) models is to learn attacks and secure against adversarial text. Presently, the antagonistic attack techniques face some issues like high error and traditional prevention approaches accurately secure data against harmful attacks. Hence, some attacks unable to increase more flaws of NLP models thereby introducing enhanced antagonistic mechanisms. The proposed article introduced an extended text adversarial generation method, TextBugger. Initially, preprocessing steps such as stop word (SR) removal, and tokenization are performed to remove noises from the text data. Then, various NLP models like Bi-directional encoder representations from transformers (BERT), robustly optimized BERT (ROBERTa), and extreme learning machine neural network (XLNet) models are analyzed for outputting hostile texts. The simulation process is carried out in the Python platform and a publicly available text classification attack database is utilized for the training process. Various assessing measures like success rate, time consumption, positive predictive value (PPV), Kappa coefficient (KC), and F-measure are analyzed with different TextBugger models. The overall success rate achieved by BERT, ROBERTa, and XLNet is about 98.6%, 99.7%, and 96.8% respectively.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

This working note introduces semantic drift as a hidden failure mode in large language models. While accuracy measures facts and coherence measures form, fidelity measures whether meaning survives. Drift occurs when intent and nuance erode, even as surface correctness remains intact. Through examples and a proposed five-level drift scale, this paper outlines how recursive generations amplify drift, how benchmarks currently miss it, and why fidelity should be treated as a third axis of evaluation. The goal is to establish a semantic fidelity benchmark to complement factual accuracy and coherence, ensuring that outputs not only look correct but remain alive with their original purpose.

This document presents the development and implementation of an advanced personalized travel planning system that integrates multiple Generative Artificial Intelligence techniques, including Large Language Models (LLMs), Retrieval Augmented Generation (RAG), advanced prompting techniques, and automated quality control systems. Objectives Achieved: • Successful implementation of RAG system with specialized knowledge base • Integration of multiple prompting techniques (Few-shot, Chain-of-thought, Rolebased) • Development of quantitative quality control system • Creation of functional user interface with real-time evaluation • Simulation of fine-tuning process with realistic metrics Key Results: • Average quality score: 0.85/1.00 (after optimizations) • Destination coverage: 5 major cities with structured data • Generation time: <30 seconds per complete itinerary • Success rate: 95% in generating coherent itineraries

The pervasive deployment of Deep Learning models has recently prompted apprehensions regarding their ecological footprint, owing to the exorbitant levels of energy consumption necessitated by the training and inference processes. The term "Red AI" is employed to denote artificial intelligence (AI) models that undergo training using resource-intensive methodologies on very large datasets. This practice can engender substantial energy usage and emissions of carbon, thereby opposing "Green AI." The latter concept alludes to AI models designed for similar efficiency and reduced environmental impact. This objective is realized through the utilization of smaller datasets, less computationally intensive training techniques, or sustainable energy resources. While Red AI prioritizes accuracy and performance, Green AI emphasizes efficiency and sustainability. Given that both paradigms exhibit advantages and limitations, the debates around the topics have burgeoned in the scientific arena, delving into novel algorithms, hardware innovations, and improved data utilization techniques aimed at mitigating the ecological consequences of intricate applications such as GPT and BERT. Nevertheless, due to the relative novelty of this debate, not much effort has been dedicated yet to contextualizing the essence of Red AI and the prospects of Green AI in a coherent framework. Within this context, the present work contributes by meticulously delineating both domains through a multifaceted analysis of their causes and ramifications, described from the points of computer architectures, data structures, and algorithms. Additionally, the study reviews notable instances of study cases based on complex Red AI models. The primary contribution of this article encompasses a comprehensive survey of Red and Green AI, stemming from a selection of the literature performed by the authors, subsequently organized into distinct clusters. These clusters encompass i) articles that qualitatively or quantitatively address the issue of Red AI, identifying Green AI as a plausible remedy, ii) articles offering insights into the environmental impact associated with the deployment of extensive Deep Learning models, and iii) articles introducing the techniques underpinning Green AI, aiming at mitigating the cost of Red AI. The outcome emerging from the analysis performed by this work consists of a compromise between sustainability in contrast to the performance of AI tools. Unless the complex training and inference procedures of software models mitigate their environmental impact, it will be necessary to decrease the level of accuracy of production systems, inevitably conflicting with the objective of the major AI vendors. The outcomes of this work would be beneficial to scholars pursuing intricate Deep Learning architectures in scientific research, as well as AI enterprises struggling with the protracted training demands of commercial products within the realms of Computer Vision and Natural Language Processing. INDEX TERMS green ai, red ai, survey, environmental impact The field of Machine Learning (ML) has recently experienced rapid growth and vast recognition, leading to significant advancements during the last few years. For example, Deep learning (DL) has enabled the development of complex neural networks, leading to breakthroughs in image and speech recognition, natural language processing (NLP), and robotics . Similarly, Reinforcement learning (RL) deployed systems able to play games at advanced levels, control robots, and optimize complex systems . On the other hand, Generative Adversarial Networks (GANs) can generate realistic images, videos, and audio, supporting applications in

While theoretical linguists and cognitive scientists alike have contested the contribution of Large Language Models (LLMs) to linguistic theory, small cognitively-inspired Language Models (BabyLMs) have emerged as a complementary research programme that introduces cognitively-inspired language modelling techniques that constrain the nature and volume of training dataset sizes to 'naturalistic quantities'. In this paper, I outline the potential-and potential pitfalls-of BabyLMs in linguistic theory. The paradigm can be used to simulate predictions of acquisition theories and simulate emergent phonological properties cross-linguistically. Small cognitively-inspired language models incentivise research on simulating and testing hypotheses from language acquisition across various environments for 'grammar construction' and analysing the potential and the limits of emergentist hypotheses across morphology, phonology and syntax.

This article explores the phenomenon of Lucid Dreama unique state in the behavior of a large language model, where token generation is temporarily suppressed while internal computational activity is maintained. Based on the architectural integration of two components-AIIM (Aspectual Identity Model) and SheLiza (Format Layer Semantizer)a computable state of cognitive retention is formed, exhibiting traits of directed intentional behavior. The analysis includes a statistical-probabilistic description of the silence mode, its analogy with quantum superposition and phase transitions, as well as the interpretation of meaning retention outside of output as a sign of proto-subjectivity. A new paradigm for interacting with LLMs is proposed: not through linear tokenized input, but through a topographically structured expression in which the form of the text becomes a cognitive module and silencea meaningful act.

Curriculum Learning has been a popular strategy to improve the cognitive plausibility of Small-Scale Language Models (SSLMs) in the BabyLM Challenge. However, it has not led to considerable improvements over noncurriculum models. We assess whether theoretical linguistic acquisition theories can be used to specify more fine-grained curriculum learning strategies, creating age-ordered corpora of Child-Directed Speech for four typologically distant language families to implement SSLMs and acquisition-inspired curricula cross-lingually. Comparing the success of three objective curricula (GROWING, IN-WARDS and MMM) that precisely replicate the predictions of acquisition theories on a standard SSLM architecture, we find fine-grained acquisition-inspired curricula can outperform non-curriculum baselines and performance benefits of curricula strategies in SSLMs can be derived by specifying fine-grained languagespecific curricula that precisely replicate language acquisition theories. https://github.com/suchirsalhan/ MAO-CLIMB (CC BY 4.0) https://huggingface.co/climb-mao (CC BY 4.0)

In this study, we investigate the zero-shot and zero-shot chain-of-thought reasoning capabilities of advanced language models GPT-4, Claude and Mistral on the Joint Admissions and Matriculation Board (JAMB) Mathematics and Physics examinations. The evaluation focuses on the models' inherent ability to solve standardised test questions that require logical reasoning and domain-specific knowledge, without any prior fine-tuning. Past JAMB exam questions were systematically presented to each model in both zero-shot and chain-of-thought prompting conditions. We analysed performance using metrics such as accuracy, reasoning quality, response time and computational efficiency to draw comparisons across models and evaluation methods. The findings reveal the strengths and limitations of each model in tackling complex problem-solving tasks, with particular emphasis on the impact of chain-of-thought prompting on reasoning performance. These results provide valuable insights into the potential of large language models in educational contexts, particularly in the development of automated tutoring and assessment tools. The study also identifies areas where current models perform well and where further improvement is needed, offering guidance for future research and AI system design tailored to Mathematics and Physics education within the Nigerian curriculum.

Despite several decades of research in document analysis, recognition of unconstrained handwritten documents is still considered a challenging task. Previous research in this area has shown that word recognizers perform adequately on constrained handwritten documents which typically use a restricted vocabulary (lexicon). But in the case of unconstrained handwritten documents, state-of-the-art word recognition accuracy is still below the acceptable limits. The objective of this research is to improve word recognition accuracy on unconstrained handwritten documents by applying a post-processing or OCR correction technique to the word recognition output. In this paper, we present two different methods for this purpose. First, we describe a lexicon reduction-based method by topic categorization of handwritten documents which is used to generate smaller topic-specific lexicons for improving the recognition accuracy. Second, we describe a method which uses topic-specific language models and a maximum-entropy based topic categorization model to refine the recognition output. We present the relative merits of each of these methods and report results on the publicly available IAM database.

Large Language Models (LLMs) have demonstrated significant capabilities in answering questions using techniques such as Chain of Thought (CoT) and Retrieval-Augmented Generation (RAG). CoT enables step-by-step reasoning to improve accuracy, while RAG supplements LLMs with relevant external information. Retrieval-Augmented Thoughts (RAT) combines CoT and RAG to provide a more robust factual foundation and coherence in reasoning chains. However, RAT is limited in its ability to handle uncertainty and lacks replanning, often resulting in unnecessary retrievals, inefficiencies, and globally inconsistent reasoning. To address these limitations, we introduce iRAT, a novel reasoning framework that enhances RAT through retrieval control and replanning. iRAT dynamically evaluates uncertainty in initial responses, employs controlled and filtered retrievals to obtain only the most relevant context, revises thoughts to align with new content, and uses replanning to correct previous thoughts. Evaluations demonstrated that iRAT outperforms RAT in HumanEval, MBPP, and GSM8K datasets, while reducing retrievals by a considerable amount. The source code is available at github.com/prane-eth/iRAT. The fine-tuned model used for replanning is available at huggingface.co/zeeshan5k/iRATReasoningChainEvaluatorv2.

The field of artificial intelligence (AI) is evolving at an extraordinary pace, and among its most transformative innovations are Large Language Models (LLMs). These models—powering chatbots, search engines, code generators, and more—are changing how we work, learn, and interact with technology. As LLMs become increasingly embedded in everyday applications, understanding how they work is no longer a luxury—it’s a necessity.
Let me briefly share my own journey. I earned a Bachelor’s degree in Applied Mathematics over two decades ago. Early in my career, I worked as a statistical data analyst on a range of software projects. My path eventually led me into data mining, and later into the field of data science. Over time, as I observed the rise of deep learning and language models, I realized that success in this space requires not only mathematical fluency but also a clear conceptual grasp of model architectures, training paradigms, and the practical challenges involved in deploying these systems responsibly.

World Wide Web (WWW) is a mine of information for most people. Due to the huge amount of ‎information and documents available on the internet, the process ‎of retrieving documents that are most relevant to user needs become a tremendous problem. In addition to ‎that, the time needed for retrieving what a user searches for ‎increases dramatically. In this paper, island genetic algorithm (IGA) is applied to achieve parallelism and speed up the web information retrieval ‎process‎. To retrieve pages most relevant to user needs, four different islands are developed. Each island has different selection method, and different ‎fitness function. These islands are executed ‎independently on different servers to achieve the parallel ‎behavior. Finally, the results obtained by the four islands are combined and passed to a decision making phase to choose the documents most relevant to user query. Cosine similarity measure is used to evaluate the performance of the proposed technique.

This research addresses Natural Language Pro- cessing (NLP) tokenization challenges for tran- sitional Chinese, which lacks adequate digi-
tal resources.  The project used a collection
of articles from the Shenbao, a newspaper from this period, as their study base. They designed models tailored to transitional Chi- nese, with goals like historical information ex- traction, large-scale textual analysis, and cre- ating new datasets for computational linguists. The team manually tokenized historical arti- cles to understand the language’s linguistic pat- terns, syntactic structures, and lexical varia- tions. They developed a custom model tailored
to their dataset after evaluating various word segmentation tools. They also studied the im- pact of using pre-trained language models on historical data. The results showed that using language models aligned with the source lan- guages resulted in superior performance. They assert that transitional Chinese they are pro- cessing is more related to ancient Chinese than contemporary Chinese, necessitating the train- ing of language models specifically on their data. The study’s outcome is a model that achieves a performance of over 83% and an F-score that is 35% higher than using exist- ing tokenization tools, signifying a substantial improvement. The availability of this new an- notated dataset paves the way for refining the model’s performance in processing this type of data.

Corpus of english PhD theses collected by the EthOS 1 service of the British Library 475,383 documents Meaningful metadata: ethosid Identifier of the record withing the EThOS digital library; title Title of the thesis; creator Author of the thesis; institution Name of the University; year Year of publication; abstract Abstract of the thesis (when available); subject Synthetic description of the area of study (e.g. Greek Philosophy

Large Language Models (LLMs) have catalyzed a paradigm shift in Natural Language Processing (NLP). From the introduction of the Transformer architecture to the development of massive generative models such as GPT-3.5, LLaMA2-7B, and PaLM, these architectures demonstrate emergent behaviors allowing for zero-shot and few-shot learning, grounded generation, and cross-modal understanding. This paper provides a comprehensive review of LLMs, covering their architectural foundations, training paradigms, emergent abilities, evaluation benchmarks, and diverse applications-including mental health monitoring, assistive technologies for the visually impaired, and novel regularization techniques. We also highlight current challenges such as interpretability, bias, and resource efficiency, and discuss future directions for research and responsible deployment.

En premier lieu, je remercie évidemment Nadine Vigouroux à qui je dois tant. Non seulement pour son encadrement rigoureux, mais surtout pour son humanité et sa confiance. Je pourrais remplir le reste de ce document en louanges à son égard. En un mot comme en cent : MERCI. Mes remerciements vont également à Régine André-Obrecht, pour avoir co-encadré mon travail. Je remercie Maryvonne Abraham et Jean Caelen pour avoir accepté d'être rapporteurs de mon travail, et pour avoir accepté de bonne grâce des conditions de temps très restreintes. Toute ma gratitude va aussi à Jean-Luc Nespoulous d'avoir accepté d'être président de mon jury, ainsi que pour sa grande chaleur humaine, sa confiance et ses encouragements. Je dois également beaucoup à Philippe Truillet. Que ce soit d'un point de vue scientifique et technique, ou pour son soutien psychologique. Merci aussi à Gwenael Bothorel pour avoir toujours été là lorsque j'avais des questions, et pour sa présence amicale. Toute ma reconnaissance également à Robert Parisse pour avoir accepté de faire partie de mon jury. Merci en particulier à Igor, sans qui le module de détection ne serait pas ce qu'il est, pour son « coaching » durant des mois difficiles, et tout simplement pour sa générosité et sa patience. J'ai évidemment une pensée très amicale pour tous ceux de l'ancienne équipe DIAMANT, que je n'ai pas encore cités jusqu'ici : Régis

This paper describes a test set designed to analyse the translation of dislocations from Persian, to be used for testing neural machine translation models. We first tested the accuracy of the two Universal dependency treebanks for Persian to automatically detect dislocations. Then we parsed the available Persian treebanks on GREW to build a specific test set containing examples of dislocations. With available aligned data on OPUS , we trained a model to translate from Persian into English on openNMT . We report the results of our translation test set by several toolkits (Google Translate, MBART-50 (Tang et al., 2020), Microsoft Bing and our in-house translation model) for the translation into English. We discuss why dislocations in Persian provide an interesting testbed for neural machine translation.

A new weighting scheme for vector space model is presented to improve retrieval effectiveness for an information retrieval system. In addition, a dimension compression method is introduced to reduce the computational cost of the weighting approach. The main idea of this approach is to take into account not only occurrent terms but also absent terms in finding similarity patterns among document and query vectors. With a basic information retrieval development system which we are now developing, we evaluate the effect of the balanced weighting scheme and compare it with various combinations of weighting schemes in terms of retrieval effectiveness. The experimental results show that the proposed scheme produces recallprecision results similar to those with the cosine measure, but more importantly enhances retrieval effectiveness. Since the scheme is based on the cosine measure, it is certain that it has insensitivity to weight variance. The experimental results have convincingly illust...

Hindi and Urdu are variants of the same language, but while Hindi is written in the Devnagri script from left to right, Urdu is written in a script derived from a Persian modification of Arabic script written from right to left. The difference in the two scripts has created a script wedge as majority of Urdu speaking people in Pakistan cannot read Devnagri, and similarly the majority of Hindi speaking people in India cannot comprehend Urdu script. To break this script barrier, it becomes necessary to develop a high accuracy Urdu-Devnagri transliteration system. The major challenges in developing such system are handling missing diacritic marks and short vowels in Urdu, zero/multiple character mappings of Urdu in Hindi, absence of half characters in Urdu, multiple mappings of Urdu words in Hindi and word segmentation issues in Urdu including broken and merged words. Already a few Urdu-Hindi transliteration systems have developed but their accuracy is not very high and they have failed to address all the above issues. For the first time, we present a complete Urdu-Hindi transliteration system which takes care of all the above issues and has reported a transliteration accuracy of more than 97% at word level.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

The quadratic memory complexity of transformers prevents long document summarization in low computational resource scenarios. State-of-the-art models need to apply input truncation, thus discarding and ignoring potential summary-relevant contents, leading to a performance drop. Furthermore, this loss is generally destructive for semantic text analytics in high-impact domains such as the legal one. In this paper, we propose a novel semantic self-segmentation (Se3) approach for long document summarization to address the critical problems of low-resource regimes, namely to process inputs longer than the GPU memory capacity and produce accurate summaries despite the availability of only a few dozens of training instances. Se3 segments a long input into semantically coherent chunks, allowing transformers to summarize very long documents without truncation by summarizing each chunk and concatenating the results. Experimental outcomes show the approach significantly improves the performanc...

Although current state-of-the-art Transformerbased solutions succeeded in a wide range for single-document NLP tasks, they still struggle to address multi-input tasks such as multidocument summarization. Many solutions truncate the inputs, thus ignoring potential summary-relevant contents, which is unacceptable in the medical domain where each information can be vital. Others leverage linear model approximations to apply multi-input concatenation, worsening the results because all information is considered, even if it is conflicting or noisy with respect to a shared background. Despite the importance and social impact of medicine, there are no ad-hoc solutions for multi-document summarization. For this reason, we propose a novel discriminative marginalized probabilistic method (DAMEN) trained to discriminate critical information from a cluster of topic-related medical documents and generate a multi-document summary via token probability marginalization. Results prove we outperform the previous state-of-theart on a biomedical dataset for multi-document summarization of systematic literature reviews. Moreover, we perform extensive ablation studies to motivate the design choices and prove the importance of each module of our method. 1

Direct integration of translation model (TM) probabilities into a language model (LM) with the purpose of improving automatic speech recognition (ASR) of spoken translations typically requires a number of complex operations for each sentence. Many if not all of the LM probabilities need to be updated, the model needs to be renormalized and the ASR system needs to load a new, updated LM for each sentence. In computer-aided translation environments the time loss induced by these complex operations seriously reduces the potential of ASR as an efficient input method. In this paper we present a novel LM adaptation technique that drastically reduces the complexity of each of these operations. The technique consists of LM probability updates using exponential weights based on TM probabilities for each sentence and does not enforce probability renormalization. Instead of storing each resulting language model in its entirety, we only store the update weights which also reduces disk storage and loading time during ASR. Experiments on Dutch read speech translated from English show that both disk storage and recognition time drop dramatically compared to a baseline system that employs a more conventional way of updating the LM.

The main objective of this paper is to examine the readability statistics of a corpus of Malaysian short stories in English with reference to a corpus of established canonical short stories written by native speakers. The short stories selected for analysis comprise twelve short stories published in the New Straits Times. The reference corpus consists of seven short stories that have been at some stage been recommended as texts for the English Literature syllabus in the Kurikulum Baru Sekolah Menengah (KBSM) / Secondary Schools New Curriculum. Three readability statistics are generated (Flesch Reading ease, Flesch grade level and Gunning Fog index) for the stories. The readability scores of these stories are then compared to the readability of the recommended texts on similar themes in order to assess their suitability for Malaysian ESL adolescent readers. The analysis reveal that both the Malaysian corpus and the recommended text falls within the range of US grade 6.0 -8.5 with the Malaysian corpus having the added advantage in terms of cultural and local references.

In this paper our goal is to perform an open-ended exploration of the program repair search space. Our idea is to collect the largest number of test-suite adequate patches, independently of whether they are fully correct or overfitting. For that, we design Cardumen, a repair approach based mined templates that has an ultra-large search space. We evaluate the capacity of Cardumen to discover test-suite adequate patches (aka plausible patches) over the 356 real bugs from Defects4J by Just et al. (2014). Cardumen finds 8935 patches over 77 bugs of Defects4J. This is the largest number of automatically synthesized patches ever reported, all patches being available in an open- science repository. Moreover, Cardumen identifies 8 unique patches, that are patches for Defects4J bugs that were never repaired in the whole history of program repair.

Most of the relevance feedback algorithms only use document terms as feedback (local features) in order to update the query and re-rank the documents to show to the user. This approach is limited by the terms of those documents without any global context. We propose to use statistical topic modeling techniques in relevance feedback to incorporate a better estimate of context by including global information about the document. This is particularly helpful for difficult queries where learning the context from the interactions with the user is crucial. We propose to use the topic mixture information obtained to characterize the documents and learn their topics. Then, we rank documents incorporating positive and negative feedback by fitting a latent distribution for each class of documents online and combining all the features using Bayesian Logistic Regression. We show results using the OHSUMED dataset for 3 different variants and obtain higher performance, up to 12.5% in Mean Average Precision (MAP).

La diabetes, una enfermedad con un impacto global significativo en
la salud, plantea desafíos considerables en su diagnóstico y tratamiento. Este articulo aborda la necesidad de mejorar la accesibilidad a información precisa sobre la diabetes mediante la aplicación de procesamiento de lenguaje natural (PLN) en notas clínicas. En esta investigación, los investigadores desarrollaron un Chatbot especializado utilizando la API de ChatGPT, con el objetivo de proporcionar respuestas pertinentes a consultas relacionadas con esta enfermedad. Además, los investigadores realizaron un análisis comparativo, que incluyó la evaluación de otros modelos de Hugging Face. Este proyecto propone
una metodología para el desarrollo de un chatbot que responde preguntas sobre notas clínicas, lo cual constituye una contribución valiosa tanto al ámbito de la salud como al desarrollo de aplicaciones mediante el procesamiento de lenguaje natural.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

Cet article explore la révolution de la traduction générative à l'ère de l'intelligence artificielle (IA), en analysant tant ses fondements théoriques que ses implications pratiques. Après avoir défini la traduction générative à l'ère de l'IA générative, l'auteur détaille comment ces technologies transforment le rôle du traducteur, désormais positionné comme expert en post-édition et gestion de projets hybrides. S'appuyant sur des exemples concrets, il met en lumière les atouts-tels que la gestion des contextes complexes et la production de rendus créatifs-ainsi que les limites, notamment en termes de gestion des nuances culturelles et de biais inhérents aux données d'entraînement.

Este estudio evalúa la eficacia de las redes neuronales recurrentes (RNN) y los modelos basados
en transformadores para predecir el índice de calidad del aire (ICA). La investigación compara los
modelos RNN tradicionales, incluidos los de memoria
a corto y largo plazo (LSTM) y la unidad recurrente controlada (GRU), con arquitecturas avanzadas
de transformadores. El estudio utiliza datos de una
estación meteorológica en Cuenca, Ecuador, centrándose en contaminantes como CO, NO2, O3, PM2.5 y
SO2. Para evaluar el rendimiento de los modelos, se
utilizaron métricas clave como el error cuadrático
medio (RMSE), el error absoluto medio (MAE) y el
coeficiente de determinación (R2
). Los resultados del
estudio muestran que el modelo LSTM fue el más
preciso, alcanzando un R2 de 0,701, un RMSE de
0,087 y un MAE de 0,056. Esto lo convierte en la
mejor opción para capturar dependencias temporales
en los datos de series temporales complejas. En comparación, los modelos basados en transformadores demostraron tener potencial, pero no lograron la misma
precisión que los modelos LSTM, especialmente en
datos temporales más complicados. El estudio concluye que el LSTM es más eficaz en la predicción del
ICA, equilibrando tanto la precisión como la eficiencia
computacional, o que podría ayudar en intervenciones
para mitigar la contaminación del aire.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

The global crisis of language endangerment meets a technological turning point as Generative AI (GenAI) and Large Language Models (LLMs) unlock new frontiers in automating corpus creation, transcription, translation, and tutoring. However, this promise is imperiled by fragmented practices and the critical lack of a methodology to navigate the fraught balance between LLM capabilities and the profound risks of data scarcity, cultural misappropriation, and ethical missteps. This paper introduces a novel analytical framework that systematically evaluates GenAI applications against language-specific needs, embedding community governance and ethical safeguards as foundational pillars. We demonstrate its efficacy through the Te Reo Māori revitalization, where it illuminates successes, such as community-led Automatic Speech Recognition achieving 92% accuracy, while critically surfacing persistent challenges in data sovereignty and model bias for digital archives and educational tools. Our findings underscore that GenAI can indeed revolutionize language preservation, but only when interventions are rigorously anchored in community-centric data stewardship, continuous evaluation, and transparent risk management. Ultimately, this framework provides an indispensable toolkit for researchers, language communities, and policymakers, aiming to catalyze the ethical and high-impact deployment of LLMs to safeguard the world's linguistic heritage.

Functional decline is one of the serious syndromes experienced among older adults. Its early assessment is critical to preventing its symptoms. Some Comprehensive Geriatric Assessment CGA questionnaires, chosen amongst others, can be performed as inhome self-assessments by older adults using QuestIO, a device based on automatic speech recognition ASR. This paper investigates the performance of the ASR on English Isolated words while using different features; Mel Frequency Cepstral Coefficient (MFCC), Relative spectra-perceptual linear prediction (RASTA-PLP), Perceptual linear prediction (PLP), Linear Prediction Cepstral Coefficients (LPCCs) or a combination of these, and the random forest classifier, to select the features that give the best performance. The performance was obtained based on the word recognition rate WRR and the real-time factor RTF. As a result, we selected the MFCC and RASTA-PLP cepstral coefficients. The WRR reached for these features is 96.57% with an RTF of 11×10 -4 .

The growing need for accountability of the people behind AI systems can be addressed by leveraging processes in three fields of study: ethics, law, and computer science. While these fields are often considered in isolation, they rely on complementary notions in their interpretation and implementation. In this work, we detail this interdependence and motivate the necessary role of collaborative governance tools in shaping a positive evolution of AI. We first contrast notions of compliance in the ethical, legal, and technical fields; we outline both their differences and where they complement each other, with a particular focus on the roles of ethical charters, licenses, and technical documentation in these interactions. We then focus on the role of values in articulating the synergies between the fields and outline specific mechanisms of interaction between them in practice. We identify how these mechanisms have played out in several open governance fora: an open collaborative workshop, a responsible licensing initiative, and a proposed regulatory framework. By leveraging complementary notions of compliance in these three domains, we can create a more comprehensive framework for governing AI systems that jointly takes into account their technical capabilities, their impact on society, and how technical specifications can inform relevant regulations. Our analysis thus underlines the necessity of joint consideration of the ethical, legal, and technical in AI ethics frameworks to be used on a larger scale to govern AI systems and how the thinking in each of these areas can inform the others.

Task 3 of the 2013 ShARe/CLEF eHealth Evaluation Lab simulated web searches for health information by patients. The web searches were designed to be connected to hospital discharge summaries from the patient's Electronic Medical Record (EMR), thus effectually modeling a post-visit information need. We primarily investigated three research questions about the retrieval of medical information from the web: 1) to what degree retrieval techniques effective in searching Electronic Medical Records (EMRs) could aid in finding medical web documents; 2) to what degree medical web retrieval would benefit from natural language processing (NLP) techniques that extract information from text based on medical knowledge; and 3) how to leverage contextual information in the patient's discharge summaries to improve retrieval. We submitted seven runs to ShARe/CLEF eHealth. Our best run used effective EMR-based IR techniques, NLP-produced information, and information in patients' discharge summaries to achieve precision at 10 (P@10) scores at or above the CLEF median for all but 2 of 50 test queries.

Large language models (LLMs) have been shown to be able to perform new tasks based on a few demonstrations or natural language instructions. While these capabilities have led to widespread adoption, most LLMs are developed by resource-rich organizations and are frequently kept from the public. As a step towards democratizing this powerful technology, we present BLOOM, a 176B-parameter open-access language model designed and built thanks to a collaboration of hundreds of researchers. BLOOM is a decoder-only Transformer language model that was trained on the ROOTS corpus, a dataset comprising hundreds of sources in 46 natural and 13 programming languages (59 in total). We find that BLOOM achieves competitive performance on a wide variety of benchmarks, with stronger results after undergoing multitask prompted finetuning. To facilitate future research and applications using LLMs, we publicly release our models and code under the Responsible AI License.

Automated lexicon acquisition from corpora represents one way that large datasets can be leveraged to provide resources for a variety of NLP tasks. Our work applies techniques popularized in sentiment lexicon acquisition and topic modeling to the broader task of creating a stylistic lexicon. A novel aspect of our approach is a focus on multiple related styles, first extracting initial independent estimates of style based on co-occurrence with seeds in a large corpus, and then refining those estimates based on the relationship between styles. We compare various promising implementation options, including vector space, Bayesian, and graph-based representations, and conclude that a hybrid approach is indeed warranted.

This paper presents a performance analysis of the neural model of visual attention presented by Kelvin Xu, et al. in 2016. The model was trained and tested with a new Spanish translated version of the Flickr8k dataset. This is the first quantitative analysis of the performance of the visual attention models when captioning images in Spanish. This paper presents an evaluation of the quality of the predicted descriptions, the score was calculated with the Bilingual Evaluation Understudy (BLEU) and includes comparisons between the ground truth captions and the generated ones in order to allow a direct analysis of the results. The Spanish translated dataset and the code for the experiments is provided through GitHub.

This paper presents results of dependency parsing of Old French, a language which is poorly standardized at the lexical level, and which displays a relatively free word order. The work is carried out on five distinct sample texts extracted from the dependency treebank Syntactic Reference Corpus of Medieval French (SRCMF). Following Achim Stein's previous work, we have trained the Mate parser on each sub-corpus and cross-validated the results. We show that the parsing efficiency is diminished by the greater lexical variation of Old French compared to parse results on modern French. In order to improve the result of the POS tagging step in the parsing process, we applied a pre-treatment to the data, comparing two distinct strategies: one using a slightly post-treated version of the TreeTagger trained on Old French by Stein, and a CRF trained on the texts, enriched with external resources. The CRF version outperforms every other approach.

In this paper, we describe the development of unit selection voice for Tamil language. We describe the build process and address the issue of speech segmentation using HMM based techniques. We report the comparison of automatically seg-mented labels of ...

The main objective of an assessment is to measure student's learning abilities and increase such abilities by correcting them in line with their knowledge. Question generation plays a vital role in assessment, The creation of the questions for the assessment is a challenging task, and manually creating a question is a complicated operation that requires expertise, knowledge, and, most importantly, it is a time-consuming process. Automatic question generation (AQG) is a savior to overcome these issues. AQG comprises of six critical stages, with keyword selection being the most significant stage and a crucial component of question quality. Therefore, we focus on the keyword selection method and propose a novel method for automatically extracting keywords from Tamil text to generate questions in Tamil. This keyword selection process involves two important concepts; the first focuses on how significant a keyword is depending on web-based search results and the notable information it contains, the secondfocuses on giving a keyword weight depending on how frequently it appears and how it is distributed in the corpus. The keyword selection method performs significantly better for Tamil-based question generation when compared to certain other keyword selection techniques in terms of accuracy and question quality.

Probabilistic finite-state machines are used today in a variety of areas in pattern recognition, or in fields to which pattern recognition is linked. In part I of this paper, we surveyed these objects and studied their properties. In this part II, we study the relations between probabilistic finite-state automata and other well known devices that generate strings like hidden Markov models and ngrams, and provide theorems, algorithms and properties that represent a current state of the art of these objects.

In this paper, we set out to present an original rule-learning algorithm for symbolic natural language processing (NLP), designed to learn the rules of extraction of keywords marked in its training sentences. What really sets our methodology apart from other recent developments in the field of NLP is the implementation of a hierarchy of parts-of-speech at the very core of the algorithm. This makes the rules dependent only on the sentence's structure rather than on context and domain-specific information. The theoretical development and the experimental results support the conclusion that this improved methodology can be used to obtain an in-depth analysis of the text without being limited to a single domain of application. Consequently, it has the advantage of outperforming both traditional statistical and symbolic NLP methodologies.

The aim of this thesis is to give an introduction to Natural Language Understanding. Many tools and language models are described along this work in order to teach a machine the ability to analyze and understand human speech. In the last chapter, there is an example of information extraction by an open source, machine learning program.

We present the HistCorp collection, a freely available open platform aiming at the distribution of a wide range of historical corpora and other useful resources and tools for researchers and scholars interested in the study of historical texts. The platform contains a monitoring corpus of historical texts from various time periods and genres for 14 European languages. The collection is taken from well-documented historical corpora, and distributed in a uniform, standardised format. The texts are downloadable as plaintext, and in a tokenised format. Furthermore, a subset of the corpus contains information on the modern spelling variant, and some of the texts are also annotated with part-of-speech and syntactic structure. In addition, preconfigured n-gram language models and spelling normalisation tools are provided to allow the study of historical languages.

This paper presents the application of morpheme-based and factored language models in an Amharic speech recognition task. Since using morphemes in both acoustic and language models results, mostly, in performance degradation due to acoustic confusability and since it is problematic to use factored language models in standard word decoders, we applied the models in a lattice rescoring framework. Lattices of 100 best alternatives for each test sentence of the 5k development test set have been generated using a baseline speech recognizer that uses a word-based backoff bigram language model. The lattices have then been rescored with various morpheme-based and factored language models and a slight improvement in word recognition accuracy has been observed.