Academia.eduAcademia.edu

Outline

Title
Abstract
Introduction
Conclusions
References
All Topics
Computer Science
Machine Learning

Cumulative Learning

Profile image of PEI WANGPEI WANG

2019, Artificial General Intelligence

https://doi.org/10.1007/978-3-030-27005-6_20
visibility

…

description

10 pages

link

1 file

Abstract

An important feature of human learning is the ability to continuously accept new information and unify it with existing knowledge, a process that proceeds largely automatically and without catastrophic side-effects. A generally intelligent machine (AGI) should be able to learn a wide range of tasks in a variety of environments. Knowledge acquisition in partially-known and dynamic task-environments cannot happen all-at-once, and AGI-aspiring systems must thus be capable of cumulative learning: efficiently making use of existing knowledge while learning new things, increasing the scope of ability and knowledge incrementally-without catastrophic forgetting or damaging existing skills. Many aspects of such learning have been addressed in artificial intelligence (AI) research, but relatively few examples of cumulative learning have been demonstrated to date and no generally accepted explicit definition exists of this category of learning. Here we provide a general definition of cumulative learning and describe how it relates to other concepts frequently used in the AI literature.

Related papers

Posthuman learning: AI from novice to expert?
Cathrine Hasse

AI & SOCIETY, 2018

Will robots ever be able to learn like humans? To answer that question, one first needs to ask: what is learning? Hubert and Stuart Dreyfus had a point when they claimed that computers and robots would never be able to learn like humans because human learning, after an initial phase of rule-based learning, is uncertain, context sensitive and intuitive (Dreyfus and Dreyfus in A five stage model of the mental activities involved in directed skill acquisition.

View PDFchevron_right
Learning to Learn
Nicholas Butko

IEEE 6th International Conference on …, 2007

View PDFchevron_right
Learning the Basics of Learning
Tihamer von Ghyczy

Entities that can learn about their environment include natural and artificial systems, acting individually or as a part of a larger group or network. In this article, we speculate whether there might be some simple mechanisms underlying some of the diverse learning strategies observed across many different fields, e.g. biology, psychology, business, machine learning, etc. We present a simple statistical model that can reproduce a wide variety of empirical learning curves, modeling performance as a function of cumulative experience.

View PDFchevron_right
Exploratory learning structures in artificial cognitive systems
Michael Felsberg

Image and Vision Computing, 2009

One major goal of the COSPAL project is to develop an artificial cognitive system architecture with the capability of exploratory learning. Exploratory learning is a strategy that allows to apply generalization on a conceptual level, resulting in an extension of competences. Whereas classical learning methods aim at best possible generalization, i.e., concluding from a number of samples of a problem class to the problem class itself, exploration aims at applying acquired competences to a new problem class. Incremental or online learning is an inherent requirement to perform exploratory learning.

View PDFchevron_right
Continual Learning in the Presence of Repetition
Gido van de Ven

arXiv (Cornell University), 2024

Continual learning (CL) provides a framework for training models in ever-evolving environments. Although re-occurrence of previously seen objects or tasks is common in real-world problems, the concept of repetition in the data stream is not often considered in standard benchmarks for CL. Unlike with the rehearsal mechanism in buffer-based strategies, where sample repetition is controlled by the strategy, repetition in the data stream naturally stems from the environment. This report provides a summary of the CLVision challenge at CVPR 2023, which focused on the topic of repetition in class-incremental learning. The report initially outlines the challenge objective and then describes three solutions proposed by finalist teams that aim to effectively exploit the repetition in the stream to learn continually. The experimental results from the challenge highlight the effectiveness of ensemble-based solutions that employ multiple versions of similar modules, each trained on different but overlapping subsets of classes. This report underscores the transformative potential of taking a different perspective in CL by employing repetition in the data stream to foster innovative strategy design.

View PDFchevron_right
Intentional Forgetting in Artificial Intelligence Systems: Perspectives and Challenges
Ingo Timm

Lecture Notes in Computer Science, 2018

Current trends, like digital transformation and ubiquitous computing, yield in massive increase in available data and information. In artificial intelligence (AI) systems, capacity of knowledge bases is limited due to computational complexity of many inference algorithms. Consequently, continuously sampling information and unfiltered storing in knowledge bases does not seem to be a promising or even feasible strategy. In human evolution, learning and forgetting have evolved as advantageous strategies for coping with available information by adding new knowledge to and removing irrelevant information from the human memory. Learning has been adopted in AI systems in various algorithms and applications. Forgetting, however, especially intentional forgetting, has not been sufficiently considered, yet. Thus, the objective of this paper is to discuss intentional forgetting in the context of AI systems as a first step. Starting with the new priority research program on 'Intentional Forgetting' (DFG-SPP 1921), definitions and interpretations of intentional forgetting in AI systems from different perspectives (knowledge representation, cognition, ontologies, reasoning, machine learning, self-organization, and distributed AI) are presented and opportunities as well as challenges are derived.

View PDFchevron_right
Intentional Forgetting: An Emerging Field in AI and Beyond
Ingo Timm

Ki - Künstliche Intelligenz, 2018

View PDFchevron_right
AI Autonomy: Self-Initiation, Adaptation and Continual Learning
Scott Grigsby

2022

As more and more AI agents are used in practice, it is time to think about how to make these agents fully autonomous so that they can (1) learn by themselves continually in a self-motivated and self-initiated manner rather than being retrained offline periodically on the initiation of human engineers and (2) accommodate or adapt to unexpected or novel circumstances. As the real-world is an open environment that is full of unknowns or novelties, detecting novelties, characterizing them, accommodating or adapting to them, and gathering ground-truth training data and incrementally learning the unknowns/novelties are critical to making the AI agent more and more knowledgeable and powerful over time. The key challenge is how to automate the process so that it is carried out continually on the agent's own initiative and through its own interactions with humans, other agents and the environment just like human on-the-job learning. This paper proposes a framework (called SOLA) for this learning paradigm to promote the research of building autonomous and continual learning enabled AI agents. To show feasibility, an implemented agent is also described.

View PDFchevron_right
A conceptual foundation for autonomous learning in unforeseen situations
Catriona Kennedy

Intelligent Control (ISIC), 1998. Held jointly with …, 1998

A cognitive agent should have the capability to learn autonomously in completely unforeseen situations. "Unforeseen" means something that was not taken into account in an internal representation of the world (i.e. it is "impossible"). However it is detectable in the form of unusual sensor measurements. Two problems must be solved: rst, the "newness" of the situation must be detected, i.e. it should not be allocated (wrongly) to an existing category. Secondly, new concepts must be learned so that when a similar situation occurs again it is no longer anomalous. A conceptual framework is presented here based on a form of symbol grounding which emphasises a continual distinction between model-driven expectancy and actual reality. Large di erences between expectation and reality indicate that a new concept is required which corresponds more accurately to the sensor data. This results in the autonomous growth and change of symbol groundings. Genetic programming is considered as a tool (both on the symbolic and the subsymbolic levels).

View PDFchevron_right
Learning Then, Learning Now, and Every Second in Between: Lifelong Learning With a Simulated Humanoid Robot
Stefan Wermter

Frontiers in Neurorobotics, 2021

Long-term human-robot interaction requires the continuous acquisition of knowledge. This ability is referred to as lifelong learning (LL). LL is a long-standing challenge in machine learning due to catastrophic forgetting, which states that continuously learning from novel experiences leads to a decrease in the performance of previously acquired knowledge. Two recently published LL approaches are the Growing Dual-Memory (GDM) and the Self-organizing Incremental Neural Network+ (SOINN+). Both are growing neural networks that create new neurons in response to novel sensory experiences. The latter approach shows state-of-the-art clustering performance on sequentially available data with low memory requirements regarding the number of nodes. However, classification capabilities are not investigated. Two novel contributions are made in our research paper: (I) An extended SOINN+ approach, called associative SOINN+ (A-SOINN+), is proposed. It adopts two main properties of the GDM model to ...

View PDFchevron_right

References (30)

  1. Aljundi, R., Kelchtermans, K., Tuytelaars, T.: Task-free continual learning. CoRR (2018)
  2. Baldassare, G., Mirolli, M., Mannella, F., Caligiore, D., Visalberghi, E., Natale, F., Truppa, V., Sabbatini, G., Guglielmelli, E., Keller, F., others: The IM-CLeVeR project: Intrinsically motivated cumulative learning versatile robots. In: 9th International Conference on Epige- netic Robotics: Modeling Cognitive Development in Robotic Systems. pp. 189-190 (2009)
  3. Bieger, J.E., Thórisson, K.R.: Task analysis for teaching cumulative learners. In: Proc. Arti- ficial General Intelligence. pp. 21-31. Springer International Publishing (2018)
  4. Caruana, R.A.: Multitask connectionist learning. In: Proceedings of the 1993 Connectionist Models Summer School. pp. 372-379 (1993)
  5. Chapelle, O., Schlkopf, B., Zien, A.: Semi-supervised learning. Adaptive Computation and Machine Learning, MIT Press, Cambridge, MA (2006)
  6. Chen, Z., Liu, B.: Lifelong Machine Learning. Morgan & Claypool Publishers (2016)
  7. Fei, G., Wang, S., Liu, B.: Learning cumulatively to become more knowledgeable. In: Pro- ceedings of the 22Nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining. pp. 1565-1574. KDD '16 (2016)
  8. Fontenla-Romero, Ó., Guijarro-Berdias, B., Martinez-Rego, D., Prez-Snchez, B., Peteiro- Barral, D.: Online machine learning. Efficiency and Scalability Methods for Computational Intellect pp. 27-54 (2013)
  9. Hammer, P., Lofthouse, T.: Goal-directed procedure learning. In: International Conference on Artificial General Intelligence. pp. 77-86. Springer (2018)
  10. Jiang, J.G., Su, Z.P., Qi, M.B., Zhang, G.F.: Multi-task coalition parallel formation strategy based on reinforcement learning. Acta Automatica Sinica 34(3), 349-352 (2008)
  11. Kirkpatrick, J., Pascanu, R., Rabinowitz, N., Veness, J., Desjardins, G., Rusu, A.A., Milan, K., Quan, J., Ramalho, T., Grabska-Barwinska, A.: Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences 114(13), 3521-3526 (2017)
  12. Lake, B., Salakhutdinov, R., Gross, J., Tenenbaum, J.: One shot learning of simple visual concepts. In: Proceedings of the Annual Meeting of the Cognitive Science Society. vol. 33 (2011)
  13. Mermillod, M., Bugaiska, A., Bonin, P.: The stability-plasticity dilemma: Investigating the continuum from catastrophic forgetting to age-limited learning effects. Front. Psychol. 4 (2013)
  14. Mitchell, T., Cohen, W., Hruschka, E., Talukdar, P., Yang, B., Betteridge, J., Carlson, A., Dalvi, B., Gardner, M., Kisiel, B., Krishnamurthy, J., Lao, N., Mazaitis, K., Mohamed, T., Nakashole, N., Platanios, E., Ritter, A., Samadi, M., Settles, B., Wang, R., Wijaya, D., Gupta, A., Chen, X., Saparov, A., Greaves, M., Welling, J.: Never-ending learning. Commun. ACM 61(5), 103-115 (2018)
  15. Nivel, E., Thórisson, K.R., Steunebrink, B.R., Dindo, H., Pezzulo, G., Rodriguez, M., Her- nandez, C., Ognibene, D., Schmidhuber, J., Sanz, R., Helgason, H.P., Chella, A., Jonsson, G.K.: Bounded Recursive Self-Improvement. Technical RUTR-SCS13006, Reykjavik Uni- versity Department of Computer Science, Reykjavik, Iceland (2013)
  16. Nivel, E., Thórisson, K.R., Dindo, H., Pezzulo, G., Rodriguez, M., Corbato, C., Steune- brink, B., Ognibene, D., Chella, A., Schmidhuber, J., Sanz, R., Helgason, H.P.: Autocat- alytic Endogenous Reflective Architecture. Technical RUTR-SCS13002, Reykjavik Univer- sity School of Computer Science, Reykjavik, Iceland (2013)
  17. Pan, S.J., Yang, Q.: A Survey on Transfer Learning. IEEE Transactions on Knowledge and Data Engineering 22(10), 1345-1359 (Oct 2010). https://doi.org/10.1109/TKDE.2009.191
  18. Ring, M.B.: CHILD: A first step towards continual learning. Machine Learning 28(1), 77- 104 (1997)
  19. Silver, D.L., Yang, Q., Li, L.: Lifelong Machine Learning Systems: Beyond Learning Algo- rithms. In: AAAI Spring Symposium: Lifelong Machine Learning (2013)
  20. Steunebrink, B.R., Thórisson, K.R., Schmidhuber, J.: Growing recursive self-improvers. In: Proceedings of Artificial General Intelligence. pp. 129-139 (2016)
  21. Taylor, M.E., Stone, P.: Transfer learning for reinforcement learning domains: A survey. J. Mach. Learn. Res. 10, 1633-1685 (2009)
  22. Thórisson, K.R., Talbot, A.: Cumulative learning with causal-relational models. In: Artificial General Intelligence. pp. 227-237. Springer International Publishing, Cham (2018)
  23. Vilalta, R., Drissi, Y.: A perspective view and survey of meta-learning. Artificial Intelligence Review 18(2), 77-95 (2002)
  24. Wang, P.: Rigid Flexibility: The Logic of Intelligence. Springer, Dordrecht (2006)
  25. Wang, P.: From NARS to a thinking machine. Advances in Artificial General Intelligence: Concepts, Architectures and Algorithms 157, 75-93 (2007)
  26. Wang, P.: Non-Axiomatic Logic: A Model of Intelligent Reasoning. World Scientific Pub- lishing, Singapore (2013)
  27. Wang, P.: Non-Axiomatic Logic: A Model of Intelligent Reasoning. World Scientific, Singa- pore (2013)
  28. Wang, P., Li, X.: Different conceptions of learning: Function approximation vs. self- organization. In: Proceedings of Artificial General Intelligence (2016)
  29. Zhan, Y., Taylor, M.E.: Online Transfer Learning in Reinforcement Learning Domains. arXiv preprint arXiv:1507.00436 (2015)
  30. Zhang, D.: From one-off machine learning to perpetual learning: A step perspective. IEEE International Conference on Systems, Man, and Cybernetics (SMC) (2018)

Related papers

Learning mechanisms for intelligent systems
Stan Franklin

2001

Abstract-In this paper, we describe mechanisms for several different types of learning being implemented in “conscious” software agents. We argue that in complex, dynamic domains, such learning mechanisms are essential for software agents to adapt and effectively 'live'in those domains. We further believe that a development period is required for knowledge acquisition.

View PDFchevron_right
Learning to learn: From smart machines to intelligent machines
Bogdan Raducanu

2008

Since its birth, more than five decades ago, one of the biggest challenges of artificial intelligence remained the building of intelligent machines. Despite amazing advancements, we are still far from having machines that reach human intelligence level. The current paper tries to offer a possible explanation of this situation. For this purpose, we make a review of different learning strategies and context types that are involved in the learning process.

View PDFchevron_right
Optimizing Reusable Knowledge for Continual Learning via Metalearning
Alvaro Soto

arXiv (Cornell University), 2021

When learning tasks over time, artificial neural networks suffer from a problem known as Catastrophic Forgetting (CF). This happens when the weights of a network are overwritten during the training of a new task causing forgetting of old information. To address this issue, we propose MetA Reusable Knowledge or MARK, a new method that fosters weight reusability instead of overwriting when learning a new task. Specifically, MARK keeps a set of shared weights among tasks. We envision these shared weights as a common Knowledge Base (KB) that is not only used to learn new tasks, but also enriched with new knowledge as the model learns new tasks. Key components behind MARK are twofold. On the one hand, a metalearning approach provides the key mechanism to incrementally enrich the KB with new knowledge and to foster weight reusability among tasks. On the other hand, a set of trainable masks provides the key mechanism to selectively choose from the KB relevant weights to solve each task. By using MARK, we achieve state of the art results in several popular benchmarks, surpassing the best performing methods in terms of average accuracy by over 10% on the 20-Split-MiniImageNet dataset, while achieving almost zero forgetfulness using 55% of the number of parameters. Furthermore, an ablation study provides evidence that, indeed, MARK is learning reusable knowledge that is selectively used by each task. Among the cognitive abilities of humans, memory is one of the most relevant. However, memories are fragile. In particular, studies from cognitive psychology show that memories can be lost when new knowledge disrupts old information, in a process known as task interference [1, 2]. In the case of artificial neural networks (ANNs), task interference is behind the problem known as Catastrophic Forgetting (CF) [3, 4, 5, 6, 7]. In effect, as a model learns a new task, its weights are overwritten causing catastrophic forgetting of old information. Unfortunately, in the case of ANNs, task interference is more severe than in the case of humans. This is evident for the case of continual learning, i.e., the case when data from new tasks is presented sequentially to the learner, who does not have access to previous data. Under this training scheme, when a previously trained model is retrained on a new task, it usually suffers a significant drop in performance in the original task [8, 9, 10]. Previous works to prevent CF have followed two main strategies. The first strategy [8, 10, 11, 12, 13] consists of avoiding the modification of parameters 35th Conference on Neural Information Processing Systems (NeurIPS 2021).

View PDFchevron_right
Learning ∈ Artificial Intelligence ∩ Cognitive Technologies ∩ Neural Computation ∩ …
Christian Igel

KI - Künstliche Intelligenz, 2012

We strive for computer systems that can deal autonomously and flexibly with our needs. Such systems must work in situations that have not been fully specified a priori. Incomplete descriptions of application scenarios are inevitable because we need software for domains where the designer's knowledge is not perfect, the solutions to particular problems are simply unknown, and/or the sheer complexity and variability of the task and the environment precludes a sufficiently accurate description of the domain. Although such systems are in general too complex to be designed manually, large amounts of data describing the task and the environment are often available or can be easily obtained. To take proper advantage of this available information, we need to develop systems that self-adapt and automatically improve based on sample data-systems that learn.

View PDFchevron_right
State-of-the-Art Techniques in Artificial Intelligence for Continual Learning: A Review
Keijo Haataja

Position and Communication Papers of the 16th Conference on Computer Science and Intelligence Systems, 2021

Artificial neural networks are used in many stateof-the-art systems for perception, and they thrive at solving classification problems, but they lack the ability to transfer that learning to a new task. Human and animals both have the capability of acquiring knowledge and transfer them continually throughout their lifespan. This term is known as continual learning. Continual learning capabilities are important to ANN in the real world especially with the continuous stream of big data. However, it remains a challenge to be achieved because they are prone to a problem called catastrophic forgetting.. Fixing this problem is critical, so that ANN incrementally learn and improve when deployed to real life situations. In this paper, we did a taxonomy of continual learning in human by introducing plasticity-stability dilemma, hence the Hebbian plasticity and compensatory homeostatic plasticity process of learning and memory formation that occurs in the brain. We also did a state-of-the-art review of three different approaches to continual learning to mitigate catastrophic forgetting.

View PDFchevron_right

Related topics

  • Computer Science
  • Machine Learning
  • Cumulative Learning
  • Constructivist AI

    • Cited by

    Sensor-Based Machine Learning Approach for Indoor Air Quality Monitoring in an Automobile Manufacturing
    Rushit Kansara

    Energies

    The alternative control concept using emission from the machine has the potential to reduce energy consumption in HVAC systems. This paper reports on a study of alternative inputs for a control system of HVAC using machine learning algorithms, based on data that are gathered in a welding area of an automotive factory. A data set of CO2, fine dust, temperatures and air velocity was logged using continuous and gravimetric measurements during two typical production weeks. The HVAC system was reduced gradually each day to trigger fluctuations of emission. The data were used to train and test various machine learning models using different statistical indices, consequently to choose a best fit model. Different models were tested and the Long Short-Term Memory model showed the best result, with 0.821 discrepancy on R2. The gravimetric samples proved that the reduction of air exchange rate does not correlate to escalation of fine dust linearly, which means one cannot rely on just gravimetr...

    View PDFchevron_right
    A Metamodel and Framework for Artificial General Intelligence From Theory to Practice
    Hugo Latapie, Jayanth Srinivasa, Özkan Kiliç, Ramana Kompella, Gaowen Liu

    Journal of Artificial Intelligence and Consciousness

    This paper introduces a new metamodel-based knowledge representation that significantly improves autonomous learning and adaptation. While interest in hybrid machine learning/symbolic AI systems leveraging, for example, reasoning and knowledge graphs, is gaining popularity, we find there remains a need for both a clear definition of knowledge and a metamodel to guide the creation and manipulation of knowledge. Some of the benefits of the metamodel we introduce in this paper include a solution to the symbol grounding problem, cumulative learning and federated learning. We have applied the metamodel to problems ranging from time series analysis, computer vision and natural language understanding and have found that the metamodel enables a wide variety of learning mechanisms ranging from machine learning, to graph network analysis and learning by reasoning engines to interoperate in a highly synergistic way. Our metamodel-based projects have consistently exhibited unprecedented accurac...

    View PDFchevron_right
    Error-Correction for AI Safety
    Leon Kester

    Artificial General Intelligence, 2020

    The complex socio-technological debate underlying safetycritical and ethically relevant issues pertaining to AI development and deployment extends across heterogeneous research subfields and involves in part conflicting positions. In this context, it seems expedient to generate a minimalistic joint transdisciplinary basis disambiguating the references to specific subtypes of AI properties and risks for an error-correction in the transmission of ideas. In this paper, we introduce a high-level transdisciplinary system clustering of ethical distinction between antithetical clusters of Type I and Type II systems which extends a cybersecurityoriented AI safety taxonomy with considerations from psychology. Moreover, we review relevant Type I AI risks, reflect upon possible epistemological origins of hypothetical Type II AI from a cognitive sciences perspective and discuss the related human moral perception. Strikingly, our nuanced transdisciplinary analysis yields the figurative formulation of the so-called AI safety paradox identifying AI control and value alignment as conjugate requirements in AI safety. Against this backdrop, we craft versatile multidisciplinary recommendations with ethical dimensions tailored to Type II AI safety. Overall, we suggest proactive and importantly corrective instead of prohibitive methods as common basis for both Type I and Type II AI safety.

    View PDFchevron_right
    Modeling with Real-Time Informative Feedback: Implementing and Evaluating a New Massive Open Online Course Component
    Hanan Kohen

    Journal of Science Education and Technology

    As part of the design, development, and deployment of a massive open online course (MOOC) on model-based systems engineering, we introduced MORTIF-Modeling with Real-Time Informative Feedback, a new learning-by-doing feature that enables the learner to model, receive detailed feedback, and resubmit improved solutions. We examined the pedagogical usability of MORTIF by investigating characteristics of participants working with it, and their perceived contribution, preferred question type, and learning style. The research included 295 participants and applied the mixed-methods approach, using MOOC server data and online questionnaires. Analyzing 12,095 submissions, we found increasing frequency of using the model resubmitting option. Students ranked MORTIF as the highest of six question types in terms of preference and perceived contribution level. Nine learning style categories were identified and classified based on students' verbal explanations regarding their preference of MORTIF over the other question types. MORTIF has been effective in promoting meaningful learning, supporting our hypothesis that the combination of active learning with real-time informative feedback is a learning mode that students eagerly embrace and benefit from. The benefits we identified for using MORTIF include active learning, provision of meaningful immediate feedback to the learner, the option to use the feedback on the spot and resubmitting an improved model, and its suitability for a variety of learning styles.

    View PDFchevron_right
    Neurosymbolic Systems of Perception and Cognition: The Role of Attention
    Hugo Latapie

    Frontiers in Psychology, 2022

    View PDFchevron_right
    580 California St., Suite 400
    San Francisco, CA, 94104
    © 2025 Academia. All rights reserved