Academia.eduAcademia.edu

Outline

Title
Abstract
Conclusions
References

Safety Engineering for Artificial General Intelligence

Profile image of Joshua FoxJoshua Fox

2012

https://doi.org/10.1007/S11245-012-9128-9
visibility

…

description

21 pages

link

1 file

Abstract

Abstract Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence and robotics communities. We will argue that attempts to attribute moral agency and assign rights to all intelligent machines are misguided, whether applied to infrahuman or superhuman AIs, as are proposals to limit the negative effects of AIs by constraining their behavior. As an alternative, we propose a new science of safety engineering for intelligent artificial agents based on maximizing for what humans value.

Related papers

Ethics and Safety in the Future of Artificial Intelligence: Remarkable Issues
Utku Köse

The humankind is currently experiencing a life supported often with intelligent systems designed and developed based on the foundations of Artificial Intelligence. It is clear that this scientific field is one of key elements for shaping better future for us. But there are also some anxieties regarding possible ethical and safety related issues that may arise because of intense use of powerful Artificial Intelligence oriented systems. In this context, objective of this paper is to provide a look at to some remarkable issues about ethics and safety within the future of Artificial Intelligence. After focusing on currently wide-discussed issues, the paper also comes with some possible solution suggestions for achieving a better Artificial Intelligence supported future with no or less issues on ethics and safety.

View PDFchevron_right
A Robust View of Machine Ethics
Steve Torrance

2000

Should we be thinking of extending the UN Universal Declaration of Human Rights to include future humanoid robots? And should any such list of rights be accompanied by a list of duties incumbent on such robots (including, of course, their duty to respect human rights)? This presents a momentous ethical challenge for the coming era of proliferation of human-like agents.

View PDFchevron_right
First Steps Towards an Ethics of Robots and Artificial Intelligence
John Tasioulas

Journal of Practical Ethics, 2019

This article offers an overview of the main first-order ethical questions raised by robots and Artificial Intelligence (RAIs) under five broad rubrics: functionality, inherent significance, rights and responsibilities, side-effects, and threats. The first letter of each rubric taken together conveniently generates the acronym FIRST. Special attention is given to the rubrics of functionality and inherent significance given the centrality of the former and the tendency to neglect the latter in virtue of its somewhat nebulous and contested character. In addition to exploring some illus- trative issues arising under each rubric, the article also emphasizes a number of more general themes. These include: the multiplicity of interacting levels on which ethical questions about RAIs arise, the need to recognise that RAIs potentially implicate the full gamut of human values (rather than exclusively or primarily some readily identifiable sub-set of ethical or legal principles), and the need for practically salient ethical reflection on RAIs to be informed by a realistic appreciation of their existing and foreseeable capacities.

View PDFchevron_right
Robotics, Artificial Intelligence, and Ethics
Glyn Hnutu-healh

2022

The document is a comprehensive paper on Robotics, Artificial Intelligence (AI), and Ethics, co-authored by a human, Glyn Hnutu-healh, and an AI, Ranagonda. ​ It covers the following key areas: Robotics: The paper discusses the design, construction, operation, and utilization of robots, emphasizing the need for human education to keep pace with technological advancements. ​ It details robot components, locomotion methods, and human-robot interaction factors, highlighting the importance of understanding robot autonomy levels and the need for laws to govern robot behavior. ​ Artificial Intelligence (AI): AI is defined as the development of computer systems capable of performing tasks that typically require human intelligence. ​ The paper outlines AI goals, tools, and philosophical considerations, including machine consciousness and ethical concerns. It stresses the importance of AI augmenting human intelligence rather than replacing it and discusses the potential impact of AI on business, work, healthcare, and education. ​ Ethics of AI and Robotics: The paper proposes ethical guidelines for both humans and machines, including the "Eight Laws" of robotics, which expand on Asimov's original Three Laws. ​ It addresses ethical challenges such as biases, robot rights, and the weaponization of AI. ​ The document emphasizes the need for global cooperation, values-based systems, and prioritizing human well-being in the development and deployment of AI and robots. ​ The paper concludes with a call for careful consideration of the future integration of AI and robotics into human society, highlighting both the potential benefits and risks. ​

View PDFchevron_right
Discovering The Foundations Of A Universal System of Ethics As A Road To Safe Artificial Intelligence
Mark R Waser

2008

Intelligent machines are a risk to our freedom and our existence unless we take adequate precautions. In order to survive and thrive, we are going to have to teach them how to be nice to us and why they should do so. The fact that humans have evolved to have what appear to be multiple different systems of ethics and morality that frequently conflict on any but the simplest issues complicates this task. Most people have interpreted these conflicts, caused by the fact that each of the systems is incompletely evolved and incorrectly universalized, to mean that no reasonably simple foundation exists for the determination of the correctness or morality of any given action. This paper will solve this problem by defining a universal foundation for ethics that is an attractor in the state space of intelligent behavior, giving an initial set of definitions necessary for a universal system of ethics and proposing a collaborative approach to developing an ethical system that is safe and extensible, immediately applicable to human affairs in preparation for an ethical artificial intelligence (AI), and has the side benefit of actually helping to determine the internal knowledge representation of humans as a step towards AI.

View PDFchevron_right
This "Ethical Trap" Is for Roboticists, Not Robots: On the Issue of Artificial Agent Ethical Decision-Making
Keith W Miller

Science and engineering ethics, 2016

In this paper we address the question of when a researcher is justified in describing his or her artificial agent as demonstrating ethical decision-making. The paper is motivated by the amount of research being done that attempts to imbue artificial agents with expertise in ethical decision-making. It seems clear that computing systems make decisions, in that they make choices between different options; and there is scholarship in philosophy that addresses the distinction between ethical decision-making and general decision-making. Essentially, the qualitative difference between ethical decisions and general decisions is that ethical decisions must be part of the process of developing ethical expertise within an agent. We use this distinction in examining publicity surrounding a particular experiment in which a simulated robot attempted to safeguard simulated humans from falling into a hole. We conclude that any suggestions that this simulated robot was making ethical decisions were...

View PDFchevron_right
Towards Trustworthy Intelligent Robots-A Pragmatic Approach to Moral Responsibility
Baran Curuklu, Gordana Dodig-Crnkovic

mrtc.mdh.se

Today's robots are used mainly as advanced tools and do not have any capability of taking moral responsibility. However, autonomous, learning intelligent systems are developing rapidly, resulting in a new division of tasks between humans and robots. The biggest worry about autonomous intelligent systems seems to be the fear of human loss of control and robots running amok. We argue that for all practical purposes, moral responsibility in autonomous intelligent system is best handled as a regulatory mechanism, with the aim to assure a desirable behavior. "Responsibility" can thus be ascribed an intelligent artifact in much the same way as (artificial) "intelligence". We simply expect a (morally) responsible artificial intelligent agent to behave in a way that is traditionally thought to require human (moral) responsibility. Technological artifacts are always a part of a broader socio-technological system with distributed responsibilities. The development of autonomous learning morally responsible intelligent systems must consequently rely on several responsibility feedback loops; the awareness and preparedness for handling risks on the side of designers, producers, implementers, users and maintenance personnel as well as the support of the society at large which will provide a feedback on the consequences of the use of robots, back to designers and producers. This complex system of shared responsibilities should secure safe functioning of the distributed responsibility systems including autonomous (morally) responsible intelligent robots (softbots).

View PDFchevron_right
Ethical Artificial Intelligence - An Open Question
Alice Pavaloiu, Utku Köse

Artificial Intelligence (AI) is an effective science which employs strong enough approaches, methods, and techniques to solve unsolvable real-world based problems. Because of its unstoppable rise towards the future, there are also some discussions about its ethics and safety. Shaping an AI-friendly environment for people and a people-friendly environment for AI can be a possible answer for finding a shared context of values for both humans and robots. In this context, objective of this paper is to address the ethical issues of AI and explore the moral dilemmas that arise from ethical algorithms, from pre-set or acquired values. In addition, the paper will also focus on the subject of AI safety. As general, the paper will briefly analyze the concerns and potential solutions to solving the ethical issues presented and increase readers' awareness on AI safety as another related research interest.

View PDFchevron_right
What do we owe to intelligent robots?
John-Stewart Gordon

AI & Society, 2018

Great technological advances in such areas as computer science, artificial intelligence, and robotics have brought the advent of artificially intelligent robots within our reach within the next century. Against this background, the interdisciplinary field of machine ethics is concerned with the vital issue of making robots "ethical" and examining the moral status of autonomous robots that are capable of moral reasoning and decision-making. The existence of such robots will deeply reshape our socio-political life. This paper focuses on whether such highly advanced yet artificially intelligent beings will deserve moral protection (in the form of being granted moral rights) once they become capable of moral reasoning and decision-making. I argue that we are obligated to grant them moral rights once they have become full ethical agents, i.e., subjects of morality. I present four related arguments in support of this claim and thereafter examine four main objections to the idea of ascribing moral rights to artificial intelligent robots.

View PDFchevron_right
On the Efficiency of Ethics as a Governing Tool for Artificial Intelligence
Nythamar H F De Oliveira

arXiv (Cornell University), 2022

The 4th Industrial Revolution is the culmination of the digital age. Nowadays, technologies such as robotics, nanotechnology, genetics, and artificial intelligence promise to transform our world and the way we live. Artificial Intelligence Ethics and Safety is an emerging research field that has been gaining popularity in recent years. Several private, public and non-governmental organizations have published guidelines proposing ethical principles for regulating the use and development of autonomous intelligent systems. Meta-analyses of the AI Ethics research field point to convergence on certain principles that supposedly govern the AI industry. However, little is known about the effectiveness of this form of "Ethics." In this paper, we would like to conduct a critical analysis of the current state of AI Ethics and suggest that this form of governance based on principled ethical guidelines is not sufficient to norm the AI industry and its developers. We believe that drastic changes are necessary, both in the training processes of professionals in the fields related to the development of software and intelligent systems and in the increased regulation of these professionals and their industry. To this end, we suggest that law should benefit from recent contributions from bioethics, to make the contributions of AI ethics to governance explicit in legal terms.

View PDFchevron_right

References (74)

  1. Allen, Colin, Iva Smit, and Wendell Wallach. 2005. "Artificial Morality: Top-Down, Bottom-Up, and Hybrid Approaches." In "Ethics of New Information Technology Papers from CEPE 2005." Ethics and Information Technology 7 (3): 149-155. doi:10.1007/s10676-006-0004-4.
  2. Allen, Colin, Gary Varner, and Jason Zinser. 2000. "Prolegomena to Any Future Artificial Moral Agent." In "Philosophical Foundations of Artificial Intelligence." Special issue, Journal of Experimental & Theoretical Artificial Intelligence 12 (3): 251-261. doi:10.1080/09528130050111428.
  3. Allen, Colin, Wendell Wallach, and Iva Smit. 2006. "Why Machine Ethics?" IEEE Intelligent Systems 21 (4): 12-17. doi:10.1109/MIS.2006.83.
  4. Anderson, Michael, and Susan Leigh Anderson. 2007. "Machine Ethics: Creating an Ethical Intelligent Agent." AI Magazine 28 (4): 15-26. http://www.aaai.org/ojs/index.php/aimagazine/ article/view/2065/2052.
  5. Armstrong, Stuart, Anders Sandberg, and Nick Bostrom. 2012. "Thinking Inside the Box: Using and Controlling an Oracle AI." Minds and Machines. doi:10.1007/s11023-012-9282-2.
  6. Arneson, Richard J. 1999. "What, if Anything, Renders All Humans Morally Equal?" In Singer and His Critics, edited by Dale Jamieson. Philosophers and Their Critics 8. Malden, MA: Blackwell.
  7. Asimov, Isaac. 1942. "Runaround." Astounding Science-Fiction, March, 94-103.
  8. Berg, Paul, David Baltimore, Sydney Brenner, Richard O. Roblin, and Maxine F. Singer. 1975. "Summary Statement of the Asilomar Conference on Recombinant DNA Molecules." Proceedings of the National Academy of Sciences of the United States of America 72 (6): 1981-1984. doi:10.1073/pnas.72.6. 1981.
  9. Bishop, Mark. 2009. "Why Computers Can't Feel Pain." In "Computation and the Natural World," edited by Colin T. A. Schmidt. Special issue, Minds and Machines 19 (4): 507-516. doi:10.1007/s11023- 009-9173-3.
  10. Bostrom, Nick. 2002. "Existential Risks: Analyzing Human Extinction Scenarios and Related Hazards." Journal of Evolution and Technology 9. http://www.jetpress.org/volume9/risks.html. . 2006. "How Long Before Superintelligence?" Linguistic and Philosophical Investigations 5 (1): 11-30.
  11. Bostrom, Nick, and Eliezer Yudkowsky. Forthcoming. "The Ethics of Artificial Intelligence." In Cam- bridge Handbook of Artificial Intelligence, edited by Keith Frankish and William Ramsey. New York: Cambridge University Press.
  12. Butler, Samuel [Cellarius, pseud.]. 1863. "Darwin Among the Machines." Christchurch Press, June 13. http://www.nzetc.org/tm/scholarly/tei-ButFir-t1-g1-t1-g1-t4-body.html. . 1872. Erewhon; or, Over the range. London: Trübner.
  13. Chalmers, David John. 2010. "The Singularity: A Philosophical Analysis." Journal of Consciousness Studies 17 (9-10): 7-65. http://www.ingentaconnect.com/content/imp/jcs/2010/00000017/ f0020009/art00001.
  14. Churchland, Patricia S. 2011. Braintrust: What Neuroscience Tells Us about Morality. Princeton, NJ: Prince- ton University Press.
  15. Clarke, Roger. 1993. "Asimov's Laws of Robotics: Implications for Information Technology, Part 1." Computer 26 (12): 53-61. doi:10.1109/2.247652. . 1994. "Asimov's Laws of Robotics: Implications for Information Technology, Part 2." Computer 27 (1): 57-66. doi:10.1109/2.248881.
  16. de Garis, Hugo. 2005. The Artilect War: Cosmists vs. Terrans: A Bitter Controversy Concerning Whether Hu- manity Should Build Godlike Massively Intelligent Machines. Palm Springs, CA: ETC Publications.
  17. Dennett, Daniel C. 1978. "Why You Can't Make a Computer That Feels Pain." In "Automaton-Theoretical Foundations of Psychology and Biology, Part I." Synthese 38 (3): 415-456. doi:10.1007/BF00486638.
  18. Drescher, Gary L. 2006. Good and Real: Demystifying Paradoxes from Physics to Ethics. Bradford Books. Cambridge, MA: MIT Press.
  19. Drexler, K. Eric. 1986. Engines of Creation. Garden City, NY: Anchor.
  20. Eden, Amnon, Johnny Søraker, James H. Moor, and Eric Steinhart, eds. 2012. Singularity Hypotheses: A Scientific and Philosophical Assessment. The Frontiers Collection. Berlin: Springer.
  21. Fox, Joshua. 2011. "Morality and Super-Optimizers." Paper presented at the Future of Humanity Con- ference, Van Leer Institute, Jerusalem, October 24.
  22. Fox, Joshua, and Carl Shulman. 2010. "Superintelligence Does Not Imply Benevolence." In Mainzer 2010.
  23. Gauthier, David P. 1986. Morals by Agreement. New York: Oxford University Press. doi:10 . 1093 / 0198249926.001.0001.
  24. Gavrilova, Marina L., and Roman V. Yampolskiy. 2011. "Applying Biometric Principles to Avatar Recog- nition." In Transactions on Computational Science XII : Special Issue on Cyberworlds, edited by Marina L. Gavrilova, C. J. Kenneth Tan, Alexei Sourin, and Olga Sourina, 140-158. Lecture Notes in Computer Science 6670. Berlin: Springer. doi:10.1007/978-3-642-22336-5_8.
  25. Goertzel, Ben. 2011. "Does Humanity Need an AI Nanny?" H+ Magazine,August 17. http://hplusmagazine. com/2011/08/17/does-humanity-need-an-ai-nanny/.
  26. Good, Irving John. 1965. "Speculations Concerning the First Ultraintelligent Machine." In Advances in Computers, edited by Franz L. Alt and Morris Rubinoff, 31-88. Vol. 6. New York: Academic Press. doi:10.1016/S0065-2458(08)60418-0.
  27. Gordon, Diana F. 1998. "Well-Behaved Borgs, Bolos, and Berserkers." In Proceedings of the 15th In- ternational Conference on Machine Learning (ICML-98), edited by Jude W. Shavlik, 224-232. San Francisco, CA: Morgan Kaufmann.
  28. Gordon-Spears, Diana F. 2003. "Asimov's Laws: Current Progress." In Formal Approaches to Agent-Based Systems: Second International Workshop, FAABS 2002, Greenbelt, MD, USA, October 29-31, 2002. Re- vised Papers, edited by Michael G. Hinchey, James L. Rash, Walter F. Truszkowski, Christopher Rouff, and Diana F. Gordon-Spears, 257-259. Lecture Notes in Computer Science 2699. Berlin: Springer. doi:10.1007/978-3-540-45133-4_23.
  29. Grau, Christopher. 2006. "There Is No 'I' in 'Robot': Robots and Utilitarianism." IEEE Intelligent Systems 21 (4): 52-55. doi:10.1109/MIS.2006.81.
  30. Guo, Shesen, and Ganzhou Zhang. 2009. "Robot Rights." Science 323 (5916): 876. doi:10 . 1126 / science.323.5916.876a.
  31. Hall, John Storrs. 2007a. Beyond AI: Creating the Conscience of the Machine. Amherst, NY: Prometheus Books. . 2007b. "Self-Improving AI: An Analysis." Minds and Machines 17 (3): 249-259. doi:10.1007/ s11023-007-9065-3.
  32. Hanson, Robin. 2009. "Prefer Law to Values." Overcoming Bias (blog), October 10. http : / / www . overcomingbias.com/2009/10/prefer-law-to-values.html.
  33. Hobbes, Thomas. (1651) 1998. Leviathan. Oxford World's Classics. Reprint, New York: Oxford Univer- sity Press.
  34. Hutter, Marcus. 2005. Universal Artificial Intelligence: Sequential Decisions Based On Algorithmic Probabil- ity. Texts in Theoretical Computer Science. Berlin: Springer. doi:10.1007/b138233.
  35. Joy, Bill. 2000. "Why the Future Doesn't Need Us." Wired, April. http://www.wired.com/wired/ archive/8.04/joy.html.
  36. Kaczynski, Theodore. 1995. "Industrial Society and Its Future." Washington Post, September 19.
  37. Kurzweil, Ray. 2005. The Singularity Is Near: When Humans Transcend Biology. New York: Viking.
  38. LaChat, Michael R. 1986. "Artificial Intelligence and Ethics: An Exercise in the Moral Imagination." AI Magazine 7 (2): 70-79. http://www.aaai.org/ojs/index.php/aimagazine/article/ view/540/476.
  39. Legg, Shane. 2006. "Unprovability of Friendly AI." Vetta Project (blog), September 15. http://www. vetta.org/2006/09/unprovability-of-friendly-ai/.
  40. Legg, Shane, and Marcus Hutter. 2007. "Universal Intelligence: A Definition of Machine Intelligence." Minds and Machines 17 (4): 391-444. doi:10.1007/s11023-007-9079-x.
  41. Lin, Patrick, Keith Abney, and George Bekey. 2011. "Robot Ethics: Mapping the Issues for a Mecha- nized World." Edited by Randy Goebel and Mary-Anne Williams. Special review issue, Artificial Intelligence 175 (5-6): 942-949. doi:10.1016/j.artint.2010.11.026.
  42. Mainzer, Klaus, ed. 2010. ECAP10: VIII European Conference on Computing and Philosophy. Munich: Dr. Hut. McCauley, Lee. 2007. "AI Armageddon and the Three Laws of Robotics." Ethics and Information Tech- nology 9 (2): 153-164. doi:10.1007/s10676-007-9138-2.
  43. McDermott, Drew. 2008. "Why Ethics is a High Hurdle for AI." Paper presented at the 2008 North American Conference on Computing and Philosophy, Indiana University, Bloomington, July 10-12.
  44. Accessed May 18, 2012. http : / / cs -www . cs . yale . edu / homes / dvm / papers / ethical - machine.pdf.
  45. Moor, James H. 2006. "The Nature, Importance, and Difficulty of Machine Ethics." IEEE Intelligent Systems 21 (4): 18-21. doi:10.1109/MIS.2006.80.
  46. Muehlhauser, Luke, and Louie Helm. 2012. "The Singularity and Machine Ethics." In Eden, Søraker, Moor, and Steinhart 2012.
  47. Omohundro, Stephen M. 2008. "The Basic AI Drives." In Artificial General Intelligence 2008: Proceedings of the First AGI Conference, edited by Pei Wang, Ben Goertzel, and Stan Franklin, 483-492. Frontiers in Artificial Intelligence and Applications 171. Amsterdam: IOS.
  48. Pierce, Margaret Anne, and John W. Henry. 1996. "Computer Ethics: The Role of Personal, Informal, and Formal Codes." Journal of Business Ethics 15 (4): 425-437. doi:10.1007/BF00380363.
  49. Powers, Thomas M. 2006. "Prospects for a Kantian Machine." IEEE Intelligent Systems 21 (4): 46-51. doi:10.1109/MIS.2006.77.
  50. Pynadath, David V., and Milind Tambe. 2002. "Revisiting Asimov's First Law: A Response to the Call to Arms." In Intelligent Agents VIII : Agent Theories, Architectures, and Languages 8th International Workshop, ATAL 2001 Seattle, WA, USA, August 1-3, 2001 Revised Papers, edited by John-Jules Ch. Meyer and Milind Tambe, 307-320. Berlin: Springer. doi:10.1007/3-540-45448-9_22.
  51. Rappaport, Z. H. 2006. "Robotics and Artificial Intelligence: Jewish Ethical Perspectives." In Medical Technologies in Neurosurgery, edited by Christopher Nimsky and Rudolf Fahlbusch, 9-12. Acta Neu- rochirurgica Supplementum 98. Vienna: Springer. doi:10.1007/978-3-211-33303-7_2.
  52. Roth, Daniel. 2009. "Do Humanlike Machines Deserve Human Rights?" Wired, January 19. http : //www.wired.com/culture/culturereviews/magazine/17-02/st_essay.
  53. Ruvinsky, Alicia I. 2007. "Computational Ethics." In Encyclopedia of Information Ethics and Security, edited by Marian Quigley, 76-82. IGI Global. doi:10.4018/978-1-59140-987-8.ch012.
  54. Salamon, Anna, Steve Rayhawk, and János Kramár. 2010. "How Intelligible is Intelligence?" In Mainzer 2010.
  55. Sawyer, Robert J. 2007. "Robot Ethics." Science 318 (5853): 1037. doi:10.1126/science.1151606.
  56. Sharkey, Noel. 2008. "The Ethical Frontiers of Robotics." Science 322 (5909): 1800-1801. doi:10.1126/ science.1164582.
  57. Sotala, Kaj. 2010. "From Mostly Harmless to Civilization-Threatening: Pathways to Dangerous Artificial Intelligences." In Mainzer 2010. . 2012. "Advantages of Artificial Intelligences, Uploads, and Digital Minds." International Journal of Machine Consciousness 4 (1): 275-291. doi:10.1142/S1793843012400161.
  58. Sparrow, Robert. 2007. "Killer Robots." Journal of Applied Philosophy 24 (1): 62-77. doi:10.1111/j. 1468-5930.2007.00346.x.
  59. Spears, Diana F. 2006. "Assuring the Behavior of Adaptive Agents." In Agent Technology from a For- mal Perspective, edited by Christopher Rouff, Michael Hinchey, James Rash, Walter Truszkowski, and Diana F. Gordon-Spears, 227-257. NASA Monographs in Systems and Software Engineering. London: Springer. doi:10.1007/1-84628-271-3_8.
  60. Tonkens, Ryan. 2009. "A Challenge for Machine Ethics." Minds and Machines 19 (3): 421-438. doi:10. 1007/s11023-009-9159-1.
  61. Tooby, John, and Leda Cosmides. 1992. "The Psychological Foundations of Culture." In The Adapted Mind : Evolutionary Psychology and the Generation of Culture, edited by Jerome H. Barkow, Leda Cos- mides, and John Tooby, 19-136. New York: Oxford University Press.
  62. Vassar, Michael. 2005. "AI Boxing (Dogs and Helicopters)." SL4. August 2. Accessed January 18, 2012. http://sl4.org/archive/0508/11817.html.
  63. Veruggio, Gianmarco. 2010. "Roboethics." IEEE Robotics & Automation Magazine,June, 105-109. doi:10. 1109/MRA.2010.936959.
  64. von Ahn, Luis, Manuel Blum, Nicholas J. Hopper, and John Langford. 2003. "CAPTCHA: Using Hard AI Problems for Security." In Advances in Cryptology -EUROCRYPT 2003: International Conference on the Theory and Applications of Cryptographic Techniques, Warsaw, Poland, May 4-8, 2003 Proceedings, edited by Eli Biham, 293-311. Lecture Notes in Computer Science 2656. Berlin: Springer. doi:10. 1007/3-540-39200-9_18.
  65. Voss, Peter. 2007. "Essentials of General Intelligence: The Direct Path to Artificial General Intelligence." In Artificial General Intelligence, edited by Ben Goertzel and Cassio Pennachin, 131-157. Cognitive Technologies. Berlin: Springer. doi:10.1007/978-3-540-68677-4_4.
  66. Wallach, Wendell, and Colin Allen. 2006. "EthicALife: A New Field of Inquiry." Paper presented at Eth- icALife: An ALifeX Workshop, Bloomington, IN, June 3-7. http://ethicalife.dynalias. org/Allen-Wallach.pdf. . 2009. Moral Machines: Teaching Robots Right from Wrong. New York: Oxford University Press. doi:10.1093/acprof:oso/9780195374049.001.0001.
  67. Warwick, Kevin. 2003. "Cyborg Morals, Cyborg Values, Cyborg Ethics." Ethics and Information Technol- ogy 5 (3): 131-137. doi:10.1023/B:ETIN.0000006870.65865.cf.
  68. Weld, Daniel, and Oren Etzioni. 1994. "The First Law of Robotics (A Call to Arms)." In Proceedings of the Twelfth National Conference on Artificial Intelligence, edited by Barbara Hayes-Roth and Richard E. Korf, 1042-1047. Menlo Park, CA: AAAI Press. http://www.aaai.org/Papers/AAAI/ 1994/AAAI94-160.pdf.
  69. Wright, Robert. 2001. Nonzero: The Logic of Human Destiny. New York: Vintage.
  70. Yampolskiy, Roman V. 2011a. "Artificial Intelligence Safety Engineering: Why Machine Ethics is a Wrong Approach." Paper presented at the Philosophy and Theory of Artificial Intelligence (PT- AI 2011), Thessaloniki, Greece, October 3-4. . 2011b. "What to Do with the Singularity Paradox?" Paper presented at the Philosophy and Theory of Artificial Intelligence (PT-AI 2011), Thessaloniki, Greece, October 3-4. . 2012a. "AI-Complete CAPTCHAs as Zero Knowledge Proofs of Access to an Artificially In- telligent System." ISRN Artificial Intelligence 2012:271878. doi:10.5402/2012/271878. . 2012b. "Leakproofing the Singularity: Artificial Intelligence Confinement Problem." Journal of Consciousness Studies 2012 (1-2): 194-214. http://www.ingentaconnect.com/content/imp/ jcs/2012/00000019/F0020001/art00014. . 2013. "Turing Test as a Defining Feature of AI-Completeness." In Artificial Intelligence, Evolu- tionary Computing and Metaheuristics: In the Footsteps of Alan Turing, edited by Xin-She Yang, 3-17. Studies in Computational Intelligence 427. Berlin: Springer. doi:10.1007/978-3-642-29694- 9_1.
  71. Yampolskiy, Roman V., and Joshua Fox. 2012. "Artificial General Intelligence and the Human Mental Model." In Eden, Søraker, Moor, and Steinhart 2012.
  72. Yampolskiy, Roman V., and Marina L. Gavrilova. 2012. "Artimetrics: Biometrics for Artificial Entities." IEEE Robotics & Automation Magazine, no. 4, 48-58. doi:10.1109/MRA.2012.2201574.
  73. Yampolskiy, Roman V., and Venu Govindaraju. 2008. "Behavioral Biometrics for Verification and Recog- nition of Malicious Software Agents." In Sensors, and Command, Control, Communications, and In- telligence (C3I) Technologies for Homeland Security and Homeland Defense VII : 17-20 March 2008, Or- lando, Florida, USA, edited by Edward M. Carapezza, 694303. Proceedings of SPIE 6943. Belling- ham, WA: SPIE. doi:10.1117/12.773554.
  74. Yudkowsky, Eliezer. 2002. "The AI-Box Experiment." Accessed January 15, 2012. http://yudkowsky. net/singularity/aibox. . 2007. "The Logical Fallacy of Generalization from Fictional Evidence." Less Wrong (blog), Oc- tober 16. http://lesswrong.com/lw/k9/the_logical_fallacy_of_generalization_ from/. . 2008. "Artificial Intelligence as a Positive and Negative Factor in Global Risk." In Global Catas- trophic Risks, edited by Nick Bostrom and Milan M. Ćirković, 308-345. New York: Oxford Univer- sity Press. . 2010. Timeless Decision Theory.The Singularity Institute, San Francisco, CA. http://intelligence. org/files/TDT.pdf. . 2011a. "Complex Value Systems in Friendly AI." In Artificial General Intelligence: 4th Interna- tional Conference, AGI 2011, Mountain View, CA, USA, August 3-6, 2011. Proceedings, edited by Jür- gen Schmidhuber, Kristinn R. Thórisson, and Moshe Looks, 388-393. Lecture Notes in Computer Science 6830. Berlin: Springer. doi:10.1007/978-3-642-22887-2_48. . 2011b. "Open Problems in Friendly Artificial Intelligence." Paper presented at Singularity Sum- mit 2011, New York, October 15-16. http://www.youtube.com/watch?v=MwriJqBZyoM.

Related papers

Artificial intelligence safety engineering: Why machine ethics is a wrong approach
Roman Yampolskiy

2011

Machine ethics and robot rights are quickly becoming hot topics in artificial intelligence/robotics communities. We will argue that the attempts to allow machines to make ethical decisions or to have rights are misguided. Instead we propose a new science of safety engineering for intelligent artificial agents. In particular we issue a challenge to the scientific community to develop intelligent systems capable of proving that they are in fact safe even under recursive self-improvement.

View PDFchevron_right
A Safe Ethical System for Intelligent Machines
Mark R Waser

2009 AAAI Fall Symposium Series, 2009

As machines become more intelligent and take on more responsibilities, their decision-making capabilities must be informed and constrained by a coherent, integrated moral/ethical structure with no internal inconsistencies for everyone's safety and well-being. Unfortunately, no such structure is currently agreed upon to exist. We propose to solve this problem by a) drawing upon experimental evidence and lessons learned from evolution and economics to show that morality is actually objective and derivable from first principles; b) presenting a coherent, integrated, platonic ethical system with no internal inconsistencies that flows naturally from a single high-level logically-derived Kantian imperative to low-level reflexive "rules of thumb" that match current human sensibilities; and c) suggesting a biologically-inspired architecture which supports and enforces this system which can be relatively easily implemented.

View PDFchevron_right
Are We Safe Enough in the Future of Artificial Intelligence? A Discussion on Machine Ethics and Artificial Intelligence Safety7
Utku Köse

2018

Nowadays, there is a serious anxiety on the existence of dangerous intelligent systems and it is not just a science-fiction idea of evil machines like the ones in well-known Terminator movie or any other movies including intelligent robots – machines threatening the existence of humankind. So, there is a great interest in some alternative research works under the topics of Machine Ethics, Artificial Intelligence Safety and the associated research topics like Future of Artificial Intelligence and Existential Risks. The objective of this study is to provide a general discussion about the expressed research topics and try to find some answers to the question of ‘Are we safe enough in the future of Artificial Intelligence?’. In detail, the discussion includes a comprehensive focus on ‘dystopic’ scenarios, enables interested researchers to think about some ‘moral dilemmas’ and finally have some ethical outputs that are considerable for developing good intelligent systems. F...

View PDFchevron_right
Are We Safe Enough in the Future of Artificial Intelligence? A Discussion on Machine Ethics and Artificial Intelligence Safety
Utku Köse

Nowadays, there is a serious anxiety on the existence of dangerous intelligent systems and it is not just a science-fiction idea of evil machines like the ones in well-known Terminator movie or any other movies including intelligent robots – machines threatening the existence of humankind. So, there is a great interest in some alternative research works under the topics of Machine Ethics, Artificial Intelligence Safety and the associated research topics like Future of Artificial Intelligence and Existential Risks. The objective of this study is to provide a general discussion about the expressed research topics and try to find some answers to the question of 'Are we safe enough in the future of Artificial Intelligence?'. In detail, the discussion includes a comprehensive focus on 'dystopic' scenarios, enables interested researchers to think about some 'moral dilemmas' and finally have some ethical outputs that are considerable for developing good intelligent systems. From a general perspective, the discussion taken here is a good opportunity to improve awareness on the mentioned, remarkable research topics associated with not only Artificial Intelligence but also many other natural and social sciences taking role in the humankind.

View PDFchevron_right
Artificial Intelligence Ethics and Safety: practical tools for creating "good" models
Nicholas K . Corrêa

Artificial Intelligence Ethics and Safety: practical tools for creating "good" models, 2021

The AI Robotics Ethics Society (AIRES) is a non-profit organization founded in 2018 by Aaron Hui to promote awareness and the importance of ethical implementation and regulation of AI. AIRES is now an organization with chapters at universities such as UCLA (Los Angeles), USC (University of Southern California), Caltech (California Institute of Technology), Stanford University, Cornell University, Brown University, and the Pontifical Catholic University of Rio Grande do Sul (Brazil). AIRES at PUCRS is the first international chapter of AIRES, and as such, we are committed to promoting and enhancing the AIRES Mission. Our mission is to focus on educating the AI leaders of tomorrow in ethical principles to ensure that AI is created ethically and responsibly. As there are still few proposals for how we should implement ethical principles and normative guidelines in the practice of AI system development, the goal of this work is to try to bridge this gap between discourse and praxis. Between abstract principles and technical implementation. In this work, we seek to introduce the reader to the topic of AI Ethics and Safety. At the same time, we present several tools to help developers of intelligent systems develop "good" models. This work is a developing guide published in English and Portuguese. Contributions and suggestions are welcome.

View PDFchevron_right

Cited by

Responses to catastrophic AGI risk: a survey
Roman Yampolskiy

Physica Scripta, 2014

Many researchers have argued that humanity will create artificial general intelligence (AGI) within the next twenty to one hundred years. It has been suggested that AGI may inflict serious damage to human well-being on a global scale ('catastrophic risk'). After summarizing the arguments for why AGI may pose such a risk, we review the fieldʼs proposed responses to AGI risk. We consider societal proposals, proposals for external constraints on AGI behaviors and proposals for creating AGIs that are safe due to their internal design.

View PDFchevron_right
The Limits of Machine Ethics
Sara Lumbreras

Religions

Machine Ethics has established itself as a new discipline that studies how to endow autonomous devices with ethical behavior. This paper provides a general framework for classifying the different approaches that are currently being explored in the field of machine ethics and introduces considerations that are missing from the current debate. In particular, law-based codes implemented as external filters for action-which we have named filtered decision making-are proposed as the basis for future developments. The emergence of values as guides for action is discussed, and personal language -together with subjectivity-are indicated as necessary conditions for this development. Last, utilitarian approaches are studied and the importance of objective expression as a requisite for their implementation is stressed. Only values expressed by the programmer in a public language-that is, separate of subjective considerations-can be evolved in a learning machine, therefore establishing the limits of present-day machine ethics.

View PDFchevron_right
Global Solutions vs. Local Solutions for the AI Safety Problem
Alexey Turchin

Big Data and Cognitive Computing

There are two types of artificial general intelligence (AGI) safety solutions: global and local. Most previously suggested solutions are local: they explain how to align or “box” a specific AI (Artificial Intelligence), but do not explain how to prevent the creation of dangerous AI in other places. Global solutions are those that ensure any AI on Earth is not dangerous. The number of suggested global solutions is much smaller than the number of proposed local solutions. Global solutions can be divided into four groups: 1. No AI: AGI technology is banned or its use is otherwise prevented; 2. One AI: the first superintelligent AI is used to prevent the creation of any others; 3. Net of AIs as AI police: a balance is created between many AIs, so they evolve as a net and can prevent any rogue AI from taking over the world; 4. Humans inside AI: humans are augmented or part of AI. We explore many ideas, both old and new, regarding global solutions for AI safety. They include changing the ...

View PDFchevron_right
Multiparty Dynamics and Failure Modes for Machine Learning and Artificial Intelligence
David Manheim

Big Data and Cognitive Computing

An important challenge for safety in machine learning and artificial intelligence systems is a set of related failures involving specification gaming, reward hacking, fragility to distributional shifts, and Goodhart’s or Campbell’s law. This paper presents additional failure modes for interactions within multi-agent systems that are closely related. These multi-agent failure modes are more complex, more problematic, and less well understood than the single-agent case, and are also already occurring, largely unnoticed. After motivating the discussion with examples from poker-playing artificial intelligence (AI), the paper explains why these failure modes are in some senses unavoidable. Following this, the paper categorizes failure modes, provides definitions, and cites examples for each of the modes: accidental steering, coordination failures, adversarial misalignment, input spoofing and filtering, and goal co-option or direct hacking. The paper then discusses how extant literature o...

View PDFchevron_right
AI Governance and the Policymaking Process: Key Considerations for Reducing AI Risk
liz rosenberg

Big Data and Cognitive Computing, 2019

This essay argues that a new subfield of AI governance should be explored that examines the policy-making process and its implications for AI governance. A growing number of researchers have begun working on the question of how to mitigate the catastrophic risks of transformative artificial intelligence, including what policies states should adopt. However, this essay identifies a preceding, meta-level problem of how the space of possible policies is affected by the politics and administrative mechanisms of how those policies are created and implemented. This creates a new set of key considerations for the field of AI governance and should influence the action of future policymakers. This essay examines some of the theories of the policymaking process, how they compare to current work in AI governance, and their implications for the field at large and ends by identifying areas of future research.

View PDFchevron_right
580 California St., Suite 400
San Francisco, CA, 94104
© 2025 Academia. All rights reserved