Anders Sandberg - 10% p(doom)? Will AI Have Weird Values? #AI #pDoom #superintelligence #AIRisk

Anders: 10% AI will wipe us out - 90% things will go swimmingly. Accelerationists often say there is no AI risks and we should rush to superintelligence, without recognising that AI may produce outcomes that are profoundly non-human. Some think AI will develop values that are alien, and that humans can't understand - but are nevertheless good ones. I'll be writing about some reflections on the problem of value arbitrariness highlighted in the classic Euthyphro dilemma, and how it relates to AI value alignment. The dilemma is found in Plato's dialogue Euthyphro where Socrates asks Euthyphro: "Is the pious (τὸ ὅσιον) loved by the gods because it is pious, or is it pious because it is loved by the gods?" Take pious to mean moral. If the former is true, even if God doesn't exist, those moral facts would still be true. An agnostic or secular AI alignment researcher considering the question of whether AI should be in alignment with values merely because humans value them, or whether AI should be in alignment with values because they are really valuable - in some grounded and/or robust sense. Put in another way, is the rightness of value independent of our attitudes? And if it's possible to do this, to what degree can we, and AI know or approximate real values, and if possible should this guide alignment efforts? Many thanks for tuning in! Please support SciFuture by subscribing and sharing! Buy me a coffee? https://buymeacoffee.com/tech101z Have any ideas about people to interview? Want to be notified about future events? Any comments about the STF series? Please fill out this form: https://docs.google.com/forms/d/1mr9PIfq2ZYlQsXRIn5BcLH2onbiSI7g79mOH_AFCdIk/ Kind regards, Adam Ford - Science, Technology & the Future - #SciFuture - http://scifuture.org