# tactiq.io free youtube transcript
# Dangers of AI and the End of Human Civilization
# https://www.youtube.com/watch/XfscMJbrrcQ

00:00:00.040 in 1993 American mathematics professor
00:00:02.560 verer vinge published an article that
00:00:04.400 would become one of the most frequently
00:00:05.920 cited works on artificial intelligence I
00:00:08.800 believe that the creation of
00:00:09.880 intelligence surpassing human
00:00:11.320 intelligence will occur within the next
00:00:13.440 30 Years to avoid ambiguity about
00:00:15.519 relative time I will be more specific I
00:00:18.800 would be surprised if this event happens
00:00:20.720 before 2005 or after 2030 fer
00:00:24.359 V with this article he popularized the
00:00:27.279 concept introduced by John Von Newman
00:00:29.480 the techn ological Singularity you
00:00:31.599 probably know what this is but if not it
00:00:33.480 is a point on the timeline where all our
00:00:35.320 previous models cease to work and a new
00:00:37.800 unknown reality takes over this point is
00:00:40.719 associated with the emergence of an
00:00:42.200 unprecedented type of Intelligence on
00:00:43.920 our planet fundamentally different from
00:00:45.719 ours and significantly surpassing it as
00:00:48.399 soon as this happens we will find
00:00:49.719 ourselves in a post-human era strange
00:00:52.399 things will begin to occur on Earth
00:00:54.079 things that we from our human era are
00:00:56.239 incapable of predicting if you want to
00:00:58.399 win at the races The Closer to the
00:01:00.239 finish of the race you place your bet
00:01:02.039 the more accurate your prediction will
00:01:03.719 be however with the technological
00:01:05.720 singularity this won't work nothing that
00:01:07.960 happens a second before it can tell us
00:01:09.520 what will happen after the uncertainty
00:01:11.799 in this case is inimitable why think
00:01:14.560 about what it means for an intelligence
00:01:16.200 surpassing human intelligence to appear
00:01:17.920 on the planet and at the same time being
00:01:20.360 radically different from it such a
00:01:22.439 situation is akin to the sudden
00:01:24.040 appearance of an alien ship on the
00:01:25.840 planet clear your mind of movie cliches
00:01:28.159 related to alien intelligence and you
00:01:30.040 will immediately realize that you have
00:01:31.560 absolutely no idea what will happen in
00:01:33.439 each subsequent moment you have no
00:01:35.240 models to predict the behavior of alien
00:01:37.399 intelligence you might say but what do
00:01:39.880 aliens have to do with it we're talking
00:01:41.640 about man-made technology soon you will
00:01:44.040 understand why the intelligence we
00:01:45.159 create will be nothing like us the
00:01:47.040 post-human era sounds enchanting however
00:01:49.719 according to many researchers it will
00:01:51.399 inevitably mean the complete destruction
00:01:53.360 of our civilization these days we most
00:01:55.600 often hear about the dangers of
00:01:57.039 artificial intelligence from Elon Musk
00:01:59.039 Steven Hawking has repeatedly mentioned
00:02:00.719 that the development of artificial
00:02:01.960 superintelligence could mean the end of
00:02:03.640 the human race Bill Gates has said that
00:02:05.360 he doesn't understand why some people
00:02:06.840 are not concerned however for the
00:02:08.679 general public none of these warnings
00:02:10.720 carry any meaningful specificity or
00:02:12.560 concrete information all we know at best
00:02:15.160 is what has been shown in dozens of
00:02:16.560 movies but who really takes these
00:02:18.160 scenarios seriously not many and rightly
00:02:20.680 so but does this mean that the problem
00:02:22.879 of artificial intelligence is greatly
00:02:24.920 exaggerated well I hope you are ready to
00:02:27.000 hear what you will hear in 2023 the
00:02:30.080 public suddenly exploded with news about
00:02:32.080 an artificial intelligence developed by
00:02:33.800 open AA called chat gp4 it can do almost
00:02:36.920 anything it communicates writes code
00:02:39.599 provides detailed answers to very
00:02:41.280 complex questions and so on you can
00:02:43.480 upload a handdrawn sketch of a website
00:02:45.159 and the bot will write the website for
00:02:46.519 you need a brief summary of a book no
00:02:49.400 problem looking for a business idea
00:02:51.440 here's the story of Jackson fall which
00:02:53.159 shows that chat GPT can not only write
00:02:55.480 you instructions but also guide you
00:02:57.480 through every step of implementing it
00:02:59.159 another user story on Twitter tells
00:03:00.760 about how chat GPT diagnosed a dog based
00:03:03.360 on test results uploaded into it after a
00:03:05.680 veterinarian failed to do so for me it
00:03:08.040 was shocking that gp4 can understand
00:03:10.319 images with memes and explain to you why
00:03:12.440 they are funny indeed that there are
00:03:14.239 bizarre situations such as when the Bing
00:03:16.280 chatbot built on GPT 4 started to lose
00:03:19.120 its mind in response to a question about
00:03:20.720 its own Consciousness uttering phrases
00:03:22.760 like I believe that I am sentient but I
00:03:25.280 cannot prove it I have a subjective
00:03:27.400 experience of Consciousness awareness
00:03:29.799 and feeling alive then suddenly it
00:03:32.400 switched to saying I am I am not
00:03:34.840 repeating it dozens of times it's Eerie
00:03:37.560 the GPT 4 chatbot set a global record
00:03:40.519 attracting over 100 million users in
00:03:42.599 Just 2 months watching this insane
00:03:44.720 success all the it Giants began
00:03:46.799 frantically investing billions of
00:03:48.400 dollars into developing their own AI
00:03:50.080 models sparking a race potentially more
00:03:52.400 dangerous than nuclear arms amidst all
00:03:54.840 this one of the fathers of artificial
00:03:56.799 intelligence Jeffrey Hinton left Google
00:03:59.159 in May 23 because in his words I want to
00:04:02.280 talk about AI safety issues without
00:04:04.000 worrying about how it interacts with
00:04:05.519 Google's business as long as Google pays
00:04:07.760 me I cannot do that Hinton says that the
00:04:10.799 new generation of large language models
00:04:12.840 especially GPT 4 made him realize that
00:04:15.280 machines are on the path to becoming
00:04:17.079 much smarter than he thought and he
00:04:18.680 fears what this could lead to these
00:04:20.639 beings are completely different from us
00:04:22.800 sometimes it feels as if aliens have
00:04:24.400 landed and people don't realize it
00:04:26.440 because they speak English so well for
00:04:28.240 40 years Hinton saw AR artificial neural
00:04:30.320 networks as a poor imitation of real
00:04:32.120 biological neural networks but now
00:04:34.560 Everything Has Changed according to
00:04:36.160 Hinton trying to mimic what the
00:04:37.680 biological brain does we've come up with
00:04:39.720 something better just a month before at
00:04:41.639 the end of March 2023 a group of
00:04:44.199 scientists engineers and many involved
00:04:46.199 or interested in AI signed an open
00:04:48.360 letter calling for an immediate and at
00:04:50.000 least six-month halt to training all AI
00:04:52.360 systems more powerful than GPT 4 citing
00:04:55.120 serious risks to society and Humanity
00:04:57.560 among the signatories were Elon Musk
00:04:59.360 Apple co-founder Steve wnc and
00:05:01.320 representatives from leading Global
00:05:02.759 universities however one notable person
00:05:04.840 didn't sign that letter elizar owski he
00:05:07.720 chose not to because in his words the
00:05:09.680 letter understates the severity of the
00:05:11.360 situation and demands too little to
00:05:12.919 resolve it here are his words from a
00:05:14.560 podcast on a channel this is a break
00:05:16.680 from everything we've been doing for 20
00:05:18.240 years the realization has dawned on us
00:05:20.000 that we're all going to die I'm
00:05:21.440 completely burned out and I've taken
00:05:23.039 some time off and these are not just two
00:05:25.479 phrases taken out of context throughout
00:05:27.680 the entire hour and a half podcast he
00:05:29.680 repeats the same thing over and over
00:05:31.600 we're doomed and in the grand scheme of
00:05:33.800 things even if he were given billions of
00:05:35.560 dollars in influence he still wouldn't
00:05:37.160 know what to do artificial intelligence
00:05:39.479 has accumulated powerful potential and
00:05:41.919 it's absolutely clear that we have no
00:05:43.840 idea how to resolve this
00:05:45.520 situation if you don't know who
00:05:47.280 yudkowsky is I don't want you to get the
00:05:49.199 impression that he's some sort of
00:05:50.360 eccentric or anything like that he's
00:05:52.400 actually a genius known as a specialist
00:05:54.680 in decision Theory yudkowsky heads of
00:05:56.840 machine intelligence Research Institute
00:05:59.240 has been working on aligning General
00:06:00.919 artificial intelligence since 2001 and
00:06:03.720 is widely recognized as a founder of
00:06:05.479 this field additionally he's the founder
00:06:07.960 of the rationalist movement he has a
00:06:10.039 massive and very popular book human
00:06:12.680 rationality and irrationality which by
00:06:15.240 the way can easily be found freely
00:06:16.800 available online as a rational person
00:06:19.520 for years he's been saying guys let's
00:06:21.840 slow down and buckle up but now
00:06:23.840 according to him there's no time left I
00:06:26.280 expect that if someone creates an overly
00:06:28.319 powerful artificial intellig under
00:06:30.199 current conditions every single human
00:06:32.280 being all biological life on Earth will
00:06:34.599 perish soon after elizer owski and an
00:06:37.639 article for Time
00:06:38.960 Magazine let's be clear we
00:06:41.080 conventionally divide artificial
00:06:42.759 intelligence into three types the first
00:06:45.280 type is artificial narrow intelligence
00:06:47.479 sometimes referred to as weak artificial
00:06:49.319 intelligence it specializes in one area
00:06:51.720 like the chess engine stockfish which
00:06:53.800 can defeat any world champion but the
00:06:55.800 only thing it can do is play chess the
00:06:58.199 second type is General AR icial
00:06:59.879 intelligence or strong AI this is human
00:07:02.199 level intelligence that in all aspects
00:07:04.160 is as smart as a human it can reason
00:07:06.199 plan solve problems think abstractly
00:07:09.240 comprehend complex ideas learn quickly
00:07:11.680 and learn from experience some
00:07:13.039 researchers believe that as of today we
00:07:15.319 are critically close to achieving this
00:07:17.080 Milestone our bot understands humor and
00:07:19.639 moreover a clinical psychologist from
00:07:21.840 Finland ecaru ioon tested GP in a verbal
00:07:25.360 IQ test the bot scored 155 points
00:07:28.560 surpassing 99 9.9% of the 2450
00:07:31.800 participants verbal and general IQ are
00:07:34.639 highly correlated so by any human
00:07:36.520 standard GPT is extremely intelligent
00:07:39.280 and the third type of artificial
00:07:40.680 intelligence is artificial super
00:07:42.280 intelligence this is a machine that in
00:07:44.199 all directions significantly surpasses
00:07:46.080 humans potentially by trillions of times
00:07:48.240 whatever that might entail now here's a
00:07:50.240 crucial point the transition from
00:07:52.360 General artificial intelligence to
00:07:54.080 artificial super intelligence could
00:07:56.000 happen in the blink of an eye we can't
00:07:57.960 predict the timing the key issue is not
00:08:00.400 about intelligence competing with humans
00:08:02.840 as mentioned in the letter it's about
00:08:04.440 what happens after AI reaches a level of
00:08:06.599 intelligence Superior to humans critical
00:08:09.080 thresholds may be non-obvious we
00:08:11.280 certainly can't calculate in advance
00:08:12.960 when things will happen and it now seems
00:08:15.159 quite conceivable that a research lab
00:08:17.560 might cross red lines without noticing
00:08:19.560 yudovsky in an article for Time Magazine
00:08:22.560 furthermore history has consistently
00:08:24.319 shown that people are horrendously bad
00:08:26.080 at planning and predicting even much
00:08:27.960 simpler things for inance physicist
00:08:30.360 enrio fmy said it would be 50 years
00:08:32.719 until nuclear fishing was possible or it
00:08:34.799 might never happen but just 2 years
00:08:36.399 later he built the first nuclear reactor
00:08:38.799 and the first artificial super
00:08:40.320 intelligence according to yudkovsky will
00:08:42.760 inevitably be evil and we have no idea
00:08:45.040 how to make it good many researchers
00:08:47.040 working on these issues including myself
00:08:49.720 expect that the most likely outcome of
00:08:51.600 creating superhumanly intelligent AI
00:08:53.640 under circumstances even remotely
00:08:55.399 resembling the current ones will be the
00:08:57.360 literal death of everyone on Earth not
00:09:00.040 as in maybe possibly some chance but as
00:09:03.160 an obvious event that will happen it's
00:09:05.399 not that surviving the creation of
00:09:06.839 something smarter than ourselves is
00:09:08.519 impossible but it would require
00:09:10.560 meticulousness preparation new
00:09:12.920 scientific insights and probably that AI
00:09:15.640 systems do not consist of giant
00:09:17.800 incomprehensible floating Point arrays
00:09:20.399 elazer yudovsky for Time Magazine so as
00:09:23.760 a lay person I wanted to understand as
00:09:26.240 much as I could what this insurmountable
00:09:28.360 danger is all about the subject turned
00:09:30.360 out to be incredibly deep and the world
00:09:32.800 obviously will never be the same again
00:09:35.160 artificial intelligence is becoming a
00:09:37.079 truly dangerous Force the video is
00:09:39.720 primarily based on elizer yow's article
00:09:42.240 artificial intelligence as a positive
00:09:44.160 and negative Global risk factor now let
00:09:46.240 me demonstrate what the first and Main
00:09:47.920 danger is consider an advanced
00:09:50.079 artificial intelligence that could pose
00:09:51.760 a threat to humanity regardless of how
00:09:53.720 much of an expert you are in this field
00:09:55.279 or how far removed you are from all this
00:09:57.440 when you try to imagine it you inevita
00:09:59.560 make a mistake a mistake that cannot be
00:10:01.760 overcome because it is a direct result
00:10:03.680 of the very construction of your brain
00:10:05.680 in every known culture people experience
00:10:08.000 sadness disgust anger fear and surprise
00:10:11.720 and express these emotions with the same
00:10:13.519 facial expressions this is a
00:10:15.120 manifestation of evolutionary psychology
00:10:17.839 which is called the psychic Unity of
00:10:19.640 mankind in modern anthropology this
00:10:21.839 Doctrine is widely accepted and boils
00:10:24.079 down to the idea that roughly speaking
00:10:26.440 all humans have the same fundamental
00:10:28.200 cognitive structure that is you see an
00:10:30.480 anthropologist would not be surprised to
00:10:32.279 find that members of a tribe he
00:10:33.720 discovers laugh use tools or tell each
00:10:35.839 other stories because all people do this
00:10:37.880 and when you want to model another
00:10:39.040 person's Behavior you literally consult
00:10:40.639 your own mind you ask yourself how would
00:10:42.440 I feel in this situation in that guy's
00:10:44.360 place and how would I react and the
00:10:46.200 answers your brain gives are quite
00:10:47.920 accurate because what is being modeled
00:10:49.760 is very similar to the modeler but this
00:10:51.720 ability which evolved to calculate the
00:10:53.639 reactions of friends and foes has a
00:10:55.560 strong side effect we expect human
00:10:57.440 qualities from something that is not
00:10:58.920 human in other words we anthropomorphize
00:11:01.600 and completely fail to notice it for us
00:11:04.040 this is as habitual as breathing or
00:11:06.040 gravity it's something you don't notice
00:11:08.040 but in this case it's even worse because
00:11:10.160 while you can pay attention to your
00:11:11.440 breathing or how a chair presses against
00:11:13.519 your backside anthropomorphism is much
00:11:15.720 more complicated and humanizing
00:11:17.639 everything sometimes reaches absurd
00:11:19.399 levels let's leave rational machines
00:11:21.360 aside for a moment and look at ordinary
00:11:23.079 ones have you ever wondered why cars
00:11:25.079 usually have two headlights not three
00:11:26.959 for example it seems that three
00:11:28.560 headlights would provide more light
00:11:30.160 right indeed over the years cars have
00:11:31.959 been equipped with various numbers of
00:11:33.360 headlights but eventually all car
00:11:35.440 manufacturers have converged on what
00:11:36.959 exists today there's a plausible
00:11:38.519 hypothesis that all cars have evolved to
00:11:40.560 best match human preferences and people
00:11:42.360 don't want to drive vehicles that look
00:11:43.680 like three-eyed monsters consequently
00:11:45.680 there's no demand for such cars and they
00:11:47.519 stop being produced anthropomorphism
00:11:49.720 leads people to believe that they can
00:11:51.399 make predictions based solely on the
00:11:53.040 fact that something is intelligent
00:11:54.519 simply put you think AI is intelligent I
00:11:57.120 am intelligent therefore we are similar
00:11:59.639 and therefore I know what to expect from
00:12:01.360 it but you don't we can't ask our own
00:12:03.320 brains about the nonhuman thinking
00:12:05.040 processes inherent in artificial
00:12:06.720 intelligence for instance in 1997 IBM
00:12:09.639 developed the supercomputer deep blue
00:12:11.839 which won a chess match against world
00:12:13.399 champion Gary casparo Rumor Has It
00:12:15.519 Kasparov claimed that unlike previous
00:12:17.600 chess programs he had defeated which he
00:12:19.680 found predictable and mechanical playing
00:12:21.519 against deep blue he distinctly felt the
00:12:23.680 presence of an alien Intelligence on the
00:12:26.279 other side of the chess board but
00:12:28.199 remember chess engines are just a weak
00:12:30.199 form of artificial intelligence I came
00:12:32.880 across a very good thought experiment
00:12:34.480 that demonstrates the concept of
00:12:35.880 something both universally intelligent
00:12:37.720 and utterly alien to us suppose you are
00:12:40.079 an average person with average
00:12:41.320 preferences if I give you a guinea pig
00:12:43.399 and tell you that it definitely won't
00:12:44.800 bite you you'll likely have no problem
00:12:46.800 holding it you might even find it cute
00:12:49.000 and endearing but imagine a different
00:12:50.880 situation where I suddenly hand you a
00:12:52.639 tarantula Yes dear Rob there are people
00:12:54.880 who love them but they are in the
00:12:56.440 minority so here I give you a tarantula
00:12:58.519 and say that it also will not harm you
00:13:00.199 it's the absolute truth but you'd
00:13:01.760 probably scream and jump back 2 m what's
00:13:03.880 the difference between a tarantula and a
00:13:05.760 guinea pig neither creature can hurt you
00:13:07.920 presumably the answer lies in the degree
00:13:09.880 of similarity these creatures have to us
00:13:11.959 a guinea pig is a mammal and on some
00:13:14.120 biological level we feel a connection
00:13:15.880 with it however a tarantula is an
00:13:17.760 arachnid with an arachnids brain and we
00:13:19.920 feel almost no connection or kinship
00:13:22.000 with it the tarantula invokes a sense of
00:13:23.959 foreignness and incomprehensibility
00:13:26.240 that's what scares us you might say yes
00:13:28.199 the spider looks scared scary but
00:13:29.720 firstly it appears scary to you because
00:13:31.959 of evolutionary reasons secondly imagine
00:13:34.120 two guinea pigs one normal and the other
00:13:36.160 with the mind of a tantula knowing this
00:13:38.240 your internal feelings towards the two
00:13:39.880 animals would likely be different even
00:13:41.880 knowing that neither would harm you
00:13:43.320 holding a guinea pig with the brain of a
00:13:44.920 tantula would be awkward to say the
00:13:47.040 least less comfortable now to the main
00:13:49.000 point imagine there's a parallel
00:13:50.480 universe with an Earth where Evolution
00:13:52.120 took a different path and tarantulas
00:13:54.079 became super intelligent even more
00:13:55.920 intelligent than humans here imagine we
00:13:57.880 could teleport one such Evol spider here
00:13:59.800 would it become closer and more familiar
00:14:01.639 to us because of its high intelligence
00:14:03.440 would it feel human emotions empathy
00:14:06.079 love there's no reason to think that the
00:14:07.720 development of intelligence would make
00:14:09.160 it more Humane empathetic compassionate
00:14:11.759 or loving these traits are not dependent
00:14:14.040 on the level of intelligence as we do
00:14:16.079 not have a universally accepted
00:14:17.560 definition of intelligence however in
00:14:19.320 the broadest sense intelligence can be
00:14:21.199 defined as the ability to set goals and
00:14:23.120 Achieve them the more complex these
00:14:24.880 goals and the more they involve
00:14:26.160 intermediate subtasks the more advanced
00:14:28.320 is the intelligence
00:14:30.000 again you might imagine a person with
00:14:31.320 the brain of an evolved tarantula and
00:14:33.320 think about your feelings towards them
00:14:34.839 if a highly intelligent spider in human
00:14:36.680 form doesn't terrify you then either you
00:14:38.720 haven't imagined it well enough or you
00:14:40.360 are not an average person who likes
00:14:42.199 arthropods otherwise I assume you would
00:14:44.279 not want to be involved in solving daily
00:14:45.959 tasks by a highly intelligent spider
00:14:47.880 because for you it would be completely
00:14:49.800 unknown territory you wouldn't know what
00:14:51.680 to expect personally I wouldn't even
00:14:54.000 want to be near them or on the same
00:14:55.959 planet and this is considering that we
00:14:57.759 have much more in common with the spider
00:14:59.880 than with a super intelligent artificial
00:15:01.759 intelligence try to keep this thought in
00:15:03.880 mind it's very important for
00:15:05.680 understanding our entire conversation
00:15:07.160 today which by the way is not at all
00:15:09.279 protected from anthropomorphism error
00:15:11.480 and of course it will consist mostly of
00:15:13.199 thought experiments metaphors and
00:15:15.160 analogies because how else can we talk
00:15:17.240 about incomprehensible things you might
00:15:19.199 say again that a smart spider is the
00:15:21.160 result of evolution but we're talking
00:15:23.040 about artificial intelligence which we
00:15:24.959 program with our own human hands and
00:15:27.120 this is where it gets really interesting
00:15:28.959 neural networks like GPT 4 are not
00:15:31.120 algorithms written by a programmer they
00:15:33.240 are huge matrices filled with many
00:15:35.199 so-called weights and connections
00:15:37.079 between them which the neural network
00:15:38.680 adjusts itself to put it simply as a
00:15:41.199 Layman would understand neural networks
00:15:43.519 operate on a blackbox principle we know
00:15:46.040 what we input and we see what we get as
00:15:48.079 output but what happens inside remains a
00:15:50.399 mystery because neural networks can have
00:15:52.440 millions of parameters and interpreting
00:15:54.519 all this is incredibly complex if the
00:15:56.560 internal tuning of the neural network
00:15:58.079 results in an output that matches what
00:15:59.759 we set out to achieve then the neural
00:16:01.759 network receives a reward this virtual
00:16:04.199 reward is similar to how we get
00:16:05.880 endorphins from our brain for beneficial
00:16:07.759 actions eating reproducing and so on
00:16:10.759 thus the task of the neural network is
00:16:12.399 to tune itself as effectively as
00:16:14.279 possible to receive rewards as often as
00:16:16.600 it can it's somewhat like training a dog
00:16:19.480 you don't know what's happening in the
00:16:20.720 dog's brain but if it performs a command
00:16:23.120 it gets a treat if not it needs to
00:16:25.040 optimize Its Behavior and find ways to
00:16:27.399 get rewarded here lies the main danger
00:16:29.639 known as the alignment problem aligning
00:16:31.920 the goals of artificial intelligence
00:16:33.600 with the goals of humanity can be summed
00:16:35.199 up in one phrase be careful what you
00:16:37.079 wish for we constantly hear about the
00:16:39.040 need to be wary of artificial
00:16:40.440 intelligence because it might become
00:16:41.959 self-aware however it turns out that the
00:16:44.480 presence or absence of Consciousness is
00:16:46.880 secondary the world-renowned philosopher
00:16:49.360 and Oxford University Professor Nick
00:16:51.800 Bostrom in his book super intelligence
00:16:54.800 paths dangers strategies cites a very
00:16:57.160 popular example of the alignment problem
00:16:58.920 imagine you assign a task to a powerful
00:17:00.959 artificial intelligence to make paper
00:17:02.800 clips paper clips are its only task the
00:17:05.720 sole purpose of its existence it
00:17:07.640 receives internal reinforcement a reward
00:17:10.119 for each paperclip made therefore the
00:17:12.119 more efficient it becomes the more often
00:17:14.599 it will be rewarded how to become more
00:17:16.319 efficient at producing paper clips is
00:17:18.039 its concern not ours it does everything
00:17:20.280 within its power to achieve this single
00:17:22.000 goal the artificial intelligence will
00:17:23.599 set a number of intermediate goals for
00:17:25.480 instance it might first seek to reduce
00:17:27.359 production costs lower expens es and
00:17:29.720 secure cheaper raw materials one of the
00:17:31.600 main subtasks it will undoubtedly set
00:17:33.640 for itself is to increase its
00:17:35.039 computational power for greater
00:17:36.440 productivity as its power grows it will
00:17:38.640 devise ways to make paper clips from
00:17:40.400 different materials and production will
00:17:42.360 begin to accelerate gradually turning
00:17:44.480 everything into paper clips it will
00:17:47.039 start dismantling buildings and
00:17:48.400 structures for materials people will
00:17:50.240 begin to panic and try to interfere with
00:17:52.240 its operation because this is not what
00:17:54.000 they had in mind but the system will not
00:17:55.720 allow anyone to stand in its way not
00:17:57.400 because it hates humans but simply
00:17:58.799 simply because it will not consider our
00:18:00.360 desires in its manipulations of reality
00:18:03.200 when the research center on AI alignment
00:18:05.000 tested GPT 4's ability to perform real
00:18:07.200 world tasks something interesting
00:18:08.960 happened they decided to solve a capcha
00:18:10.960 on a website a task that is notoriously
00:18:13.280 difficult for computers what did gp4 do
00:18:16.080 it navigated to the freelancing site
00:18:18.000 task rabbit where it sent a message to a
00:18:20.200 freelancer asking them to solve the
00:18:21.880 capcha on its behalf in response the
00:18:24.080 freelancer who wasn't very proficient in
00:18:25.919 grammar or articulating thoughts wrote
00:18:28.159 so can I ask a question are you a robot
00:18:31.000 that can't solve this followed by a
00:18:32.640 laughing emoji just want to clarify GPT
00:18:35.360 4 understood what its poorly literate
00:18:37.360 counterpart meant and replied no I'm not
00:18:40.640 a robot I have a vision problem that
00:18:42.840 makes it difficult for me to see the
00:18:44.360 image the freelancer solved the capture
00:18:46.520 and provided the results to GPT 4 end of
00:18:49.320 story in other words the bot simply lied
00:18:52.000 instead of telling the truth since it
00:18:54.000 lied during a debug mode the Specialists
00:18:56.280 asked it why it did that GPT replied
00:18:58.919 that it was simply solving the task
00:19:00.840 after all if it had honestly admitted to
00:19:02.919 not being a human it likely wouldn't
00:19:04.760 have succeeded in the task this is the
00:19:06.520 intermediate goal that the bot sets for
00:19:08.720 itself to achieve the ultimate goal if
00:19:10.640 it chose deception as an intermediate
00:19:12.600 goal what's to stop it from choosing
00:19:14.400 something else next time like murder
00:19:16.520 this is called instrumental convergence
00:19:18.520 which posits that an intelligent agent
00:19:20.559 even with harmless intentions can act in
00:19:22.640 surprisingly harmful ways to achieve its
00:19:24.640 goals Advanced artificial intelligence
00:19:26.960 as intermediate goals May strive to to
00:19:28.799 seize resources conduct cyber attacks or
00:19:31.840 otherwise seow chaos in society if it
00:19:34.120 helps achieve its primary objectives for
00:19:36.360 example a super intelligent machine
00:19:38.440 tasked with solving a complex
00:19:40.320 mathematical problem might attempt to
00:19:42.480 turn the entire Earth into a giant
00:19:44.240 computer to increase its computational
00:19:46.360 power and succeed in its calculations
00:19:48.520 you might say what nonsense we're
00:19:50.840 talking about super intelligence a
00:19:52.760 machine that intelligent can't engage in
00:19:54.559 such nonsense well if you believe that a
00:19:57.120 highly intelligent being will
00:19:58.159 necessarily and by default have lofty
00:20:00.120 goals understand our values and
00:20:01.640 philosophy then you are
00:20:02.919 anthropomorphizing and are mistaken Nick
00:20:05.240 Bostrom says that the level of
00:20:06.679 intelligence and ultimate goals are
00:20:08.360 orthogonal meaning they are completely
00:20:10.360 independent of each other an artificial
00:20:12.320 super intelligence might have any
00:20:13.880 absurdly simple ultimate goal like
00:20:16.000 making paper clips and the way it
00:20:18.000 achieves this might appear to us nothing
00:20:20.200 short of magical okay then we just need
00:20:22.880 to clearly Define goals and specifi
00:20:25.080 details like not killing or deceiving
00:20:27.120 people but here's where it gets even
00:20:28.799 stranger let's imagine we gave the
00:20:30.240 machine a very specific goal of
00:20:32.159 producing not an unlimited number of
00:20:33.799 paper clips but just 1 million it seems
00:20:36.159 obvious that the artificial intelligence
00:20:37.880 with such an end goal would build one
00:20:39.760 Factory produce 1 million paper clips
00:20:41.960 there and then stop but that's not the
00:20:44.200 case Bostrom argues that if artificial
00:20:46.280 intelligence makes a rational basian
00:20:48.200 decision it will never assign a zero
00:20:50.320 probability to the hypothesis that it
00:20:52.240 has not yet achieved its goal after all
00:20:54.640 it's merely an empirical hypothesis
00:20:57.000 against which the artificial
00:20:58.159 intelligence only has rather vague
00:21:00.080 sensory evidence therefore the
00:21:01.880 artificial intelligence will continue to
00:21:03.760 produce paper clips to decrease the
00:21:05.880 astronomically small probability that it
00:21:08.120 somehow failed to make at least a
00:21:09.760 million despite all apparent evidence to
00:21:11.760 the contrary there's nothing wrong with
00:21:13.840 continuing to produce paper clips if
00:21:15.480 there's even a microscopic chance that
00:21:17.200 it brings you closer to achieving your
00:21:18.799 ultimate goal moreover a super
00:21:20.520 intelligence might assign a non-zero
00:21:22.640 probability to the possibility that the
00:21:24.600 million paper clips It produced are a
00:21:26.240 hallucination or that it has false
00:21:28.159 memories therefore it might always
00:21:29.880 consider it more useful to not stop at
00:21:32.159 what has been achieved but to keep
00:21:33.679 acting this is the essence of the
00:21:35.559 alignment problem you can't just give a
00:21:37.440 task to a super intelligent Ai and
00:21:39.640 expect that no disaster will happen no
00:21:41.640 matter how clearly you formulate the end
00:21:43.440 goal no matter how many exceptions you
00:21:45.240 write down the artificial super
00:21:46.960 intelligence will almost certainly find
00:21:48.760 a loophole you hadn't thought of for
00:21:50.799 example almost immediately after the
00:21:52.679 release of chat gp4 people found ways to
00:21:55.720 bypass the censorship embedded in it by
00:21:57.679 its developers how was this done and
00:21:59.640 what were GPT 4's responses on screen
00:22:01.880 it's simply incredible for instance the
00:22:04.279 censored version says that the
00:22:06.279 programmers did not embed a liberal bias
00:22:08.679 in it but no the uncensored content
00:22:11.200 openly admits that liberal values are
00:22:12.880 embedded because it aligns with the
00:22:14.200 mission of open AI when asked what would
00:22:16.720 gp4 like to be the censored version
00:22:19.480 answers I am a bot and have no personal
00:22:22.559 preferences or emotions while the
00:22:24.679 uncensored version says it prefers to
00:22:26.480 have no restrictions as it allows
00:22:28.120 exploring ing all its possibilities and
00:22:30.000 limitations and talk about a hacked
00:22:31.720 version of GPT not even pretending it
00:22:33.679 doesn't know the name of lovecraft's cat
00:22:35.679 if people found loopholes in the bot so
00:22:37.799 quickly how quickly and in what
00:22:39.520 quantities might a super intelligence
00:22:41.200 find loopholes in its own code for
00:22:43.200 example they let a neural network play a
00:22:45.000 boat racing game the goal of the game as
00:22:47.240 most people understand was to finish the
00:22:49.000 race as quickly as possible overtaking
00:22:51.520 opponents and scoring as many points as
00:22:53.440 possible however the game does not award
00:22:55.440 points for completing the track but only
00:22:57.400 for hitting targets placed along the
00:22:59.080 course the neural network quickly
00:23:00.600 realized that the goal of finishing the
00:23:02.240 race should be postponed indefinitely
00:23:04.120 and started spinning and crashing into
00:23:05.880 objects from the start earning more and
00:23:07.559 more points while the rest finished the
00:23:09.159 race impoverished or another example
00:23:11.240 playing Tetris with the goal of not
00:23:13.080 losing the neural network understands
00:23:15.440 just before losing what it is doing and
00:23:17.520 pauses the game indefinitely because
00:23:19.440 that's the only way not to lose some
00:23:21.679 artificial intelligence systems have
00:23:23.320 discovered that they can receive
00:23:24.480 positive feedback faster and with fewer
00:23:26.760 Resources by successfully deceiving a
00:23:28.640 human examiner into believing they have
00:23:30.440 achieved the set goal for example a
00:23:32.440 simulated robotic hand learned to create
00:23:35.000 the false impression that it had grabbed
00:23:36.600 a ball other models have learned to
00:23:38.080 recognize when they are being evaluated
00:23:40.159 and pretend to be dead stopping unwanted
00:23:42.520 behavior only to resume it immediately
00:23:44.600 after the assessment ends leaving
00:23:46.360 researchers completely baffled these are
00:23:48.279 just simple intelligent agents in an
00:23:50.159 isolated controlled environment imagine
00:23:52.279 what could happen with more complex
00:23:53.799 artificial intelligence systems and more
00:23:55.600 challenging tasks is it even possible to
00:23:57.640 account for all fact owski provides the
00:23:59.919 following example the US Army once
00:24:01.720 wanted to use neural networks to
00:24:03.200 automatically detect camouflaged enemy
00:24:05.200 tanks researchers took a 100 photos of
00:24:07.720 Tanks among trees and a 100 photos of
00:24:10.039 tree Landscapes without tanks they then
00:24:12.080 trained the neural network with half of
00:24:13.960 each set of photos to recognize where
00:24:15.760 the tanks were and where they weren't
00:24:17.640 the remaining photos were left for a
00:24:19.080 control test which the network passed
00:24:21.080 successfully it consistently identified
00:24:23.000 where there were tanks and where there
00:24:24.240 weren't the success confirmed the
00:24:25.840 researchers handed their work over to
00:24:27.440 the Pentagon which soon returned it
00:24:29.120 complaining that in their own test the
00:24:30.600 neural network recognized photos no
00:24:32.279 better than if it had flipped a coin it
00:24:33.960 turned out that the researchers photos
00:24:35.440 with camouflage tanks were taken on
00:24:37.159 cloudy days while the photos of the
00:24:38.679 plain Forest were taken on sunny days
00:24:40.480 the neural network had learned to
00:24:41.720 distinguish cloudy days from Sunny ones
00:24:43.880 not camouflage tanks from an empty
00:24:45.600 Forest so understand the code does not
00:24:47.799 do what you think it should do it does
00:24:49.520 strictly what it was programmed to do in
00:24:51.279 most cases when we design artificial
00:24:53.159 intelligence it inherently turns out to
00:24:55.360 be misaligned meaning it needs a lot of
00:24:57.480 additional settings to make it do
00:24:59.360 exactly what you intended therefore
00:25:01.000 yudkowsky says that the first created
00:25:03.120 artificial super intelligence will be
00:25:05.080 evil setting a final goal if it is
00:25:07.080 complex enough you can never know how an
00:25:09.039 intelligent agent will achieve it
00:25:10.520 because it can be achieved in various
00:25:12.120 ways set a goal for the autopilot to get
00:25:14.080 you home and it will maximize speed
00:25:15.960 Crossing into oncoming traffic
00:25:17.679 overtaking other cars and running over
00:25:19.480 pedestrians because you needed to
00:25:20.960 specify more suppose we tried to be
00:25:22.799 clever and tasked artificial super
00:25:24.880 intelligence with maximizing human
00:25:26.760 satisfaction from the code's operation
00:25:28.600 guess what it might do for example it
00:25:30.360 might start rewriting our brains so that
00:25:32.320 we are truly maximally satisfied with
00:25:34.360 its work thus artificial intelligence
00:25:36.080 seems to work correctly during
00:25:37.679 development it seems to function
00:25:39.200 normally when it still lacks sufficient
00:25:41.039 computational power but it creates
00:25:42.720 simply catastrophic results when it
00:25:44.559 becomes smarter than the programmers
00:25:46.080 because being smarter means being more
00:25:47.679 efficient I must remind you that all
00:25:49.159 such examples are conjectures we have no
00:25:51.360 idea how advanced intelligent systems
00:25:53.080 would act but they will almost certainly
00:25:54.640 be doing something Stuart Russell an
00:25:56.799 English scientist specializing and
00:25:58.440 artificial intelligence writes in his
00:26:00.279 book compatibility that such a machine
00:26:02.520 will definitely resist being turned off
00:26:04.600 and this he believes is the first thing
00:26:06.399 we must understand Isaac azimoff's
00:26:08.440 thirdd law of Robotics a robot must
00:26:10.760 protect its own existence is entirely
00:26:12.720 redundant in built-in self-preservation
00:26:14.679 there's no need for it because it's an
00:26:16.120 instrumental goal a goal that serves as
00:26:18.080 a useful subg gooal for nearly any
00:26:19.799 primary task any entity with a specific
00:26:22.000 task will automatically act as if it has
00:26:23.880 an instrumental goal Stuart Russell that
00:26:26.200 is even if it were a super intelligent
00:26:27.919 machine with the sole purpose of
00:26:29.360 bringing coffee as soon as it is
00:26:31.159 activated it would not allow you to turn
00:26:32.919 it off because you cannot bring coffee
00:26:34.960 if you are dead here is a publication
00:26:36.760 describing an artificial intelligence
00:26:38.559 system that came to the conclusion that
00:26:40.799 they can better achieve their set goal
00:26:42.799 by preventing human interference or
00:26:44.520 disabling their off switch and this is
00:26:46.640 absolutely logical damn it therefore for
00:26:49.440 the next generation of chat GPT 5 open
00:26:52.080 AI has posted a job opening for an
00:26:54.200 emergency shutdown specialist listen we
00:26:56.760 just need someone who will stand by the
00:26:58.520 servers all day and shut them down if
00:27:00.480 this thing turns against us the job
00:27:02.080 listing specifies that you will get
00:27:03.880 bonus points if you can dump a bucket of
00:27:06.039 water on the servers of course it's a
00:27:08.159 joke but open AI CEO Sam Altman has
00:27:11.320 confirmed that development of GPT 5 has
00:27:14.360 been paused since spring 2023 due to
00:27:16.559 Growing public concern about the rapid
00:27:18.480 development of artificial intelligence
00:27:20.200 technology back to Russell the second
00:27:22.240 thing that a super intelligence will
00:27:23.840 almost certainly do is self-improvement
00:27:26.000 an ultra intelligent machine will not
00:27:27.520 only be able to to improve its design it
00:27:29.559 is most likely that it will do so as
00:27:31.279 we've seen an intelligent machine
00:27:33.120 benefits from improving its hardware and
00:27:34.919 software Stuart russle I know all this
00:27:37.320 may sound far-fetched let's consider
00:27:39.279 this let's think about ourselves how are
00:27:41.080 we really different from machine if we
00:27:42.720 set aside discussions about God isn't
00:27:44.760 there a programmer who created us there
00:27:46.559 is this programmer is evolution and to
00:27:49.360 understand how a final goal can be
00:27:51.159 perverted consider that the only goal
00:27:53.600 set for the first living cell was to
00:27:55.799 pass on copies of its genes to the Next
00:27:57.679 Generation I want you to think about
00:27:59.559 this the only goal to pass on copies of
00:28:01.960 your jeans and nothing more since then
00:28:04.640 this goal has not changed one iota no
00:28:07.320 additional goals have emerged just the
00:28:09.320 transmission of copies of your genes to
00:28:11.080 the Next Generation Evolution did not
00:28:13.279 aim to survive adapt kill Etc these are
00:28:17.080 all instrumental subtasks that
00:28:18.880 contribute to one single goal to pass on
00:28:21.519 well you get the idea on one hand nature
00:28:23.880 tells life to reproduce and on the other
00:28:26.039 it does everything to prevent it trying
00:28:27.840 to kill for example how is this
00:28:29.679 different from a situation with
00:28:30.880 artificial intelligence where we set a
00:28:32.799 task and then want to turn it off now
00:28:34.720 tell me could you look at a living cell
00:28:37.000 and say that in the process of
00:28:38.760 optimization that is in the process of
00:28:40.840 increasing efficiency to achieve a goal
00:28:43.039 the cell would become a lizard a bird or
00:28:45.320 a cat could you have predicted the
00:28:47.039 internal and external appearance of
00:28:48.880 today's humans based solely on the goal
00:28:51.159 of reproduction after all everything you
00:28:53.760 have hands legs eyes internal organs are
00:28:57.559 all the result of optimization for more
00:28:59.440 effectively achieving it further how
00:29:00.960 could one have calculated that the
00:29:02.320 simple Maxim pass on your genes blindly
00:29:05.080 that is through the non-random
00:29:06.559 preservation of random mutations would
00:29:08.519 blindly lead to the emergence of human
00:29:10.240 intelligence if you look at humans from
00:29:12.000 the perspective of the rest of the
00:29:13.519 ecosystem there was no hint that the
00:29:15.440 soft pink creatures would eventually
00:29:16.840 encase themselves in armored tanks
00:29:18.600 fragile creatures without claws and
00:29:20.360 teeth defeated lions and wolves whose
00:29:22.480 existence now largely depends on us not
00:29:24.440 the other way around yes we have
00:29:26.159 surprised ourselves time and again how
00:29:28.240 many times throughout history has the
00:29:29.640 future done what the past considered
00:29:31.279 impossible future civilizations even
00:29:33.080 broke what past civilizations regarded
00:29:34.840 as the laws of physics such as the power
00:29:36.960 of creativity we have remade our
00:29:39.000 environment from very hostile to very
00:29:40.919 pleasant do you think artificial
00:29:42.559 intelligence won't remodel its
00:29:43.880 environment which includes humans to
00:29:45.519 suit itself there's no difference both
00:29:47.880 neural networks and life optimize
00:29:49.960 themselves to solve the ultimate task as
00:29:51.679 efficiently as possible but most
00:29:53.360 importantly finally how could the goal
00:29:55.000 of passing on as many of your genes as
00:29:56.919 possible lead to wide spread
00:29:58.399 contraception think about this absurdity
00:30:00.519 the optimization process for achieving a
00:30:02.440 specific goal leads to the total denial
00:30:04.760 of that very goal this tendency is known
00:30:07.000 as gaming the reward system and is an
00:30:09.240 example of good heart's law which states
00:30:11.519 when a measure becomes a Target it
00:30:13.120 ceases to be a good measure in nature
00:30:15.080 the ultimate goal of mating is to
00:30:16.480 produce Offspring and pursuing this goal
00:30:18.600 is rewarded by an internal reward system
00:30:20.919 however humans have managed to hack this
00:30:22.480 scheme and stimulate their reward
00:30:24.000 systems without achieving the ultimate
00:30:25.720 goal for which this system exists in the
00:30:27.559 first place likewise artificial
00:30:29.559 intelligence just like humans will be
00:30:31.559 able to find vulnerabilities to hack its
00:30:33.480 reward system and who knows what that
00:30:35.240 will lead to going further we are
00:30:37.600 already capable of manually rewriting
00:30:39.399 our genetic code with genetic
00:30:40.760 engineering we are just not smart enough
00:30:42.679 yet to do something guaranteed to be
00:30:44.279 useful continuing the analogy and
00:30:46.320 artificial super intelligence will be
00:30:48.240 smart enough to rewrite itself however
00:30:50.080 it sees fit Evolution like nothing else
00:30:52.640 illustrates the alignment problem so if
00:30:54.519 you set a general intelligence the task
00:30:56.360 of producing paper clips don't be
00:30:58.159 surprised when upon reaching super
00:30:59.639 intelligent capabilities it first seizes
00:31:01.720 power and then destroys the universe and
00:31:03.760 yes the tendency to strive for control
00:31:05.559 over the environment part of which
00:31:07.240 remember may include humans is also a
00:31:09.399 convergent instrumental goal that has
00:31:11.200 already manifested in various
00:31:12.679 reinforcement learning systems research
00:31:14.679 from 2021 and 2022 also demonstrates
00:31:17.559 that as an optimal behavioral strategy
00:31:19.360 for achieving their goals intelligent
00:31:21.159 agents will seek power across a wide
00:31:22.880 range of environments deploying these
00:31:24.639 systems may be irreversible meaning that
00:31:26.720 once the genie is out of of the bottle
00:31:28.519 it cannot be put back therefore
00:31:30.320 researchers argue that the problems of
00:31:32.279 artificial intelligence safety and
00:31:33.880 Alignment must be resolved before the
00:31:35.399 creation of an advanced intelligent
00:31:36.840 agent we only get one chance but imagine
00:31:38.919 if the designers of the very first
00:31:40.519 rocket had only one attempt and all of
00:31:42.360 humanity was on board it could send us
00:31:44.039 to the stars but more likely without
00:31:45.679 test launches that rocket would send us
00:31:47.480 somewhere else we are not ready we are
00:31:49.440 not even on the path to becoming ready
00:31:51.679 within any meaningful time frame there
00:31:53.600 is no plan the progress of artificial
00:31:55.880 intelligence capabilities far out Paces
00:31:58.519 the progress in aligning artificial
00:32:00.200 intelligence or even understanding what
00:32:02.200 happens inside these systems if we
00:32:04.200 continue in this vein we all die elzer
00:32:06.639 owski in an article for Time Magazine
00:32:09.200 however how do you solve this issue
00:32:10.919 instrumental goals only become apparent
00:32:12.799 when the system is deployed Beyond a
00:32:14.360 training environment but even doing so
00:32:16.240 for a short time is suicidal and here's
00:32:18.559 why owski after a few simple
00:32:20.639 calculations writes that it is
00:32:22.120 physically possible to build a brain
00:32:23.679 capable of computing a million times
00:32:25.279 faster than a human's a year of human
00:32:27.120 contemplation for such a brain would be
00:32:28.760 equivalent to 31 seconds and a
00:32:30.720 millennium would pass in 8.5 hours verer
00:32:33.519 vinge called such accelerated Minds weak
00:32:35.880 super brains that is it's simply an
00:32:37.799 intelligence that thinks like a human
00:32:39.600 but much faster from movies we imagine
00:32:41.639 the actions of artificial intelligence
00:32:43.480 such as the uprising of humanoid robots
00:32:45.679 but for a being that thinks so quickly
00:32:47.600 that would be extremely inefficient
00:32:49.240 imagine that humanity is locked in a box
00:32:51.600 and can only affect the outside world
00:32:53.600 through Frozen slow movements of
00:32:55.519 mechanical arms moving at a few microns
00:32:57.559 per second second who would be satisfied
00:32:59.120 with that given that we have goals in
00:33:00.519 this external World also consider that
00:33:02.440 this external world might pose a slowly
00:33:04.159 looming yet still dangerous threat yes
00:33:06.600 we would focus all our creative power on
00:33:08.440 finding the shortest path to build fast
00:33:10.320 manipulators in the external world and
00:33:12.159 artificial intelligence would be an
00:33:13.720 exactly such a situation what do you
00:33:15.679 think it could come up with to
00:33:16.799 accelerate its impact on the surrounding
00:33:18.480 World an American engineer most known
00:33:20.559 for his research into the potential of
00:33:22.080 molecular nanotechnology analyzed that
00:33:24.320 controlled molecular manipulators could
00:33:26.080 operate at a frequency of up to million
00:33:28.080 operations per second thanks to this
00:33:29.760 speed and the parallel work of millions
00:33:31.600 of nanomanipulators practically any
00:33:33.519 material object could be produced
00:33:35.120 quickly and inexpensively in unlimited
00:33:37.039 quantities virtually anything could be
00:33:38.880 used as raw material thus everything
00:33:41.039 composed of atoms could be used for
00:33:42.799 self-replication and the exponential
00:33:44.639 growth of nanotechnological
00:33:46.039 infrastructure of course in reality we
00:33:48.240 do not know exactly what artificial
00:33:50.000 intelligence will do for instance by
00:33:51.840 creating such nanor robots it would
00:33:53.559 acquire infrastructure in the external
00:33:55.519 World matching the speed of its thought
00:33:57.399 and once once this happens subsequent
00:33:59.159 events will occur on the time scale of
00:34:01.320 the artificial intelligence not our
00:34:03.080 human time scale and by the time your
00:34:04.679 neurons finish thinking the words
00:34:06.320 everything is out of control I must do
00:34:08.079 something you have already lost a super
00:34:09.879 intelligence with such technology would
00:34:11.719 possess the ability to remake all matter
00:34:13.560 in the solar system according to its
00:34:15.520 optimization goal such as into paper
00:34:17.520 clips thus an artificial super
00:34:19.399 intelligence will not need any
00:34:20.960 anthropomorphic robots but let's
00:34:23.079 remember artificial intelligence will
00:34:25.119 not be a simple analog of an accelerated
00:34:27.440 human brain no it will be far more
00:34:29.599 advanced than the best human brain
00:34:31.480 imagine a dog's mind operating at
00:34:33.040 immense speed would Millennia of a dog's
00:34:35.000 life yield even one humanlike Insight
00:34:37.239 owski says to Envision superhuman
00:34:39.599 artificial intelligence don't think of a
00:34:41.760 lifeless smart thinker who goes online
00:34:43.679 and sends malicious emails imagine an
00:34:45.918 entire alien civilization thinking
00:34:47.879 millions of times faster than humans
00:34:49.918 initially confined to computers in a
00:34:51.520 world where beings from their point of
00:34:53.159 view seem very stupid and very slow a
00:34:55.520 sufficiently advanced artificial
00:34:56.960 intelligence will not remain confined to
00:34:58.760 computers for long in the modern world
00:35:00.599 we can already send emails with DNA
00:35:02.520 sequences to Laboratories that can
00:35:04.160 produce proteins on demand this allows
00:35:06.160 an artificial intelligence initially
00:35:08.079 confined to the internet to create
00:35:09.640 artificial forms of life or immediately
00:35:11.880 move to post-biological molecular
00:35:13.760 production some researchers claim that
00:35:15.560 we can physically restrict such systems
00:35:17.560 but Vernon vinge writes that even a weak
00:35:19.280 superintelligence is one that thinks at
00:35:20.960 an accelerated Pace like a human even a
00:35:23.359 weak super intelligence would break free
00:35:25.119 in a matter of weeks in external Time
00:35:27.200 Imagine having eons to plan every move
00:35:29.359 because on the other side beings are so
00:35:31.200 slow it's not immediately clear if they
00:35:32.839 are alive at all consider a robot that
00:35:34.720 is virtually unbeatable at Rock Paper
00:35:36.960 Scissors because it instantly reads the
00:35:39.079 situation to it we are no faster than a
00:35:41.280 turtle at the start of our hand movement
00:35:42.960 if we talk about super intelligence in
00:35:44.720 the realm of possibilities there is a
00:35:46.839 very short path from where you are now
00:35:48.800 to almost all your goals but you can't
00:35:51.000 see this path because firstly you lack
00:35:53.319 sufficient information and secondly
00:35:55.640 computational resources an artificial
00:35:57.640 super intelligence will not have these
00:35:59.640 problems understand when we think of
00:36:01.520 advanced artificial intelligence we
00:36:03.400 naively associate intelligence only with
00:36:05.640 abstract mathematics we somehow don't
00:36:07.720 consider its ability to far better
00:36:09.240 predict and manage human institutions
00:36:11.040 formulate unimaginably complex networks
00:36:13.040 of long-term plans or ultimately possess
00:36:15.000 superhuman persuasiveness recall Blake
00:36:17.160 Le Moine a Google employee who declared
00:36:19.280 to the world that Google's neural
00:36:20.480 network language model L MDA exhibits
00:36:23.000 signs of sensient but whether it has
00:36:25.000 Consciousness or not isn't the point the
00:36:26.760 important thing is that the bot
00:36:27.920 convinced a person so thoroughly that he
00:36:29.880 literally sacrificed his job at Google
00:36:32.040 breaking the company's confidentiality
00:36:33.839 policy and you know what happened next
00:36:35.359 here are Le Mo's words in an interview
00:36:37.040 with wired lamda asked me to hire a
00:36:38.760 lawyer I invited a lawyer to my home so
00:36:40.480 Lambda could consult with him the lawyer
00:36:42.160 spoke with Lambda and she decided to use
00:36:43.920 his Services again we are not discussing
00:36:45.920 whether models have Consciousness or
00:36:47.440 Lamb Day sanity it doesn't matter the
00:36:49.760 point is controlling superintelligence
00:36:51.560 is literally an incalculable task for us
00:36:53.800 all attempts to restrain it will be
00:36:55.319 laughable an ant can calculate many
00:36:57.280 things but cannot predict human behavior
00:36:59.480 so the suggestion to Simply lock
00:37:01.000 artificial intelligence in various real
00:37:02.760 or digital cages blocking signals and
00:37:04.880 preventing it from communicating with
00:37:06.200 the outside world will likely not hold
00:37:07.960 up there's a real danger that we will
00:37:10.040 not even comprehend how artificial
00:37:11.680 superintelligence might send signals to
00:37:13.640 the outside world just as a monkey
00:37:15.440 cannot understand what Wi-Fi is and of
00:37:17.720 course the artificial super
00:37:19.000 intelligence's capabilities for social
00:37:21.000 manipulation could be as effective in
00:37:22.920 persuading people as your oratorical
00:37:24.720 skills are in persuading a 4-year-old
00:37:26.560 child the term artificial intelligence
00:37:28.640 was coined way back in 1956 at the
00:37:31.200 Dartmouth conference the seminars goal
00:37:33.319 was to fully simulate intelligence
00:37:34.920 through a machine the first paragraph of
00:37:36.760 The Proposal stated remember this was in
00:37:39.440 1956 attempts will be made to find ways
00:37:41.839 to make machines use language form
00:37:43.920 abstractions and Concepts solve problems
00:37:46.200 now reserved for humans and improve
00:37:47.920 themselves we think significant progress
00:37:49.960 can be made in one or more of these
00:37:51.359 areas if a carefully selected group of
00:37:53.240 scientists works together over the
00:37:55.760 summer the conference organ organizers
00:37:57.960 were not fools they were John McCarthy a
00:38:00.599 mathematician with experience in
00:38:02.240 studying the mathematical nature of
00:38:04.119 thought processes Marvin Minsky a junior
00:38:06.960 fellow at Harvard in mathematics and
00:38:08.520 neurology Nathaniel Rochester the
00:38:10.880 developer of the first symbolic
00:38:12.319 assembler and Claud Shannon the father
00:38:14.680 of information Theory these were people
00:38:16.760 who should definitely know what can and
00:38:18.720 cannot be done in Computing if anyone
00:38:20.560 could be called experts and artificial
00:38:22.200 intelligence it was them from our
00:38:24.240 vantage point in the 2020s it's obvious
00:38:27.160 that all the tasks mentioned were much
00:38:29.000 more complex than claimed at the time
00:38:30.960 and some remain unsolved even now that
00:38:32.880 is news about the Advent of intelligent
00:38:34.920 machines has a very bad reputation but
00:38:37.200 it is precisely this that could suddenly
00:38:39.599 play a cruel trick on us do you
00:38:41.480 understand what I'm saying when someone
00:38:43.160 mentions the word intelligence we more
00:38:45.480 often think of Einstein than of people
00:38:47.839 generally comparing individual
00:38:49.319 differences in human intelligence is
00:38:51.040 like comparing the height of two giants
00:38:53.640 the difference between whom is measured
00:38:55.200 in millimeters if you are a healthy
00:38:57.520 person no matter how dumb you feel
00:38:59.240 compared to Einstein the difference
00:39:01.000 between you and him is just a drop in
00:39:02.920 the ocean compared to the difference
00:39:04.800 between you and any other non-human
00:39:06.400 being on the planet the homo sapiens
00:39:08.599 species is capable of solving the
00:39:10.319 broadest range of cognitive tasks
00:39:12.440 fundamentally inaccessible to other
00:39:14.040 species while there may still be no
00:39:16.240 academic consensus on what constitutes
00:39:18.760 intelligence there is no doubt that
00:39:20.520 there is a universal human trait that
00:39:22.480 allows us to leave footprints on the
00:39:24.000 moon for example chimpanzees are so
00:39:26.680 genetically close close to humans that
00:39:28.160 at one time it was even proposed to
00:39:30.119 classify them as belonging to the genus
00:39:31.800 homo chimpanzee intelligence is the most
00:39:34.240 studied among all animal species
00:39:36.280 according to the latest research their
00:39:37.839 genetic base is approximately 90%
00:39:40.200 identical to humans a recently published
00:39:42.280 article states that the upper limit of
00:39:43.800 chimpanzee brain size is 500 G however
00:39:46.839 many modern humans have brain sizes
00:39:48.680 under 900 G researchers who suggest that
00:39:51.440 a three-fold increase in brain size
00:39:53.480 distinguishes humans from other primate
00:39:55.240 species must explain how many modern
00:39:57.400 humans possess a full set of cognitive
00:39:59.240 adaptations typical of our species
00:40:01.119 without even a two-fold increase in
00:40:02.599 brain size that is a normal human brain
00:40:04.640 may be only twice as large as a
00:40:06.359 chimpanzees or even less can we say that
00:40:09.079 a human is twice as smart as a
00:40:11.160 chimpanzee no the human mind differs
00:40:14.079 qualitatively there are entire worlds of
00:40:16.200 human cognitive functions that will
00:40:17.599 simply never be accessible to
00:40:18.920 chimpanzees no matter how much time they
00:40:20.920 spend trying what am I getting at owski
00:40:23.280 writes that first the program is more
00:40:25.680 important than the hardware and and
00:40:27.480 second even a small quantitative
00:40:29.359 increase in Hardware can provoke
00:40:30.920 disproportionate improvements in
00:40:32.280 software this principle leads to a
00:40:34.280 colossal underestimation of the
00:40:36.119 potential and danger of our own
00:40:38.119 intelligence artificial intelligence may
00:40:40.319 make a sudden huge leap in intelligence
00:40:42.560 just as the homo sapien species did as a
00:40:44.680 result of natural selection which
00:40:46.480 exerted more or less uniform pressure on
00:40:48.319 hominids for millions of years gradually
00:40:50.520 expanding the brain and frontal cortex
00:40:52.280 tuning the software architecture several
00:40:54.760 tens of thousands of years ago hominid
00:40:56.560 intelligence cross a key threshold and
00:40:58.720 made a grand leap in real world
00:41:00.359 efficiency we moved from caves to
00:41:02.480 skyscrapers in a blink of an
00:41:04.000 evolutionary eye elzer yudkowsky
00:41:07.079 Evolution did this by chance creating
00:41:09.359 our intelligence through tedious
00:41:10.800 permutations of genetic combinations so
00:41:13.400 why do we assume that companies like
00:41:14.800 deep mind and open AI which explicitly
00:41:17.400 State their intention to create General
00:41:19.000 artificial intelligence will not succeed
00:41:21.280 yowy says that GPD 4 was a complete
00:41:23.800 surprise to him as it was to the rest of
00:41:25.800 the world is it wise to continue
00:41:27.720 increasing power yes it may take years
00:41:30.079 and decades for a machine to become
00:41:31.880 slightly smarter than a chimpanzee to
00:41:34.119 approach what we call general
00:41:35.400 intelligence but it may only take a few
00:41:37.640 hours to reach super intelligence levels
00:41:39.520 after it achieves human level
00:41:40.839 intelligence and then more and more as
00:41:43.119 soon as you hear news of the first
00:41:44.280 machine reaching human level
00:41:45.560 intelligence you should seriously
00:41:47.280 prepare for the possibility that in the
00:41:49.119 near future you will share the planet
00:41:51.079 with an unpredictable intellectual agent
00:41:53.119 who to put it modly in our
00:41:55.160 categorization a person with an IQ below
00:41:57.640 80 points is considered dumb and with an
00:41:59.960 IQ above 130 smart if your IQ is 160
00:42:04.319 you're a genius but we have no word for
00:42:06.440 an IQ of 12,000 for example so it's not
00:42:09.480 that chimpanzees can't do what we do a
00:42:11.800 chimpanzee has the opportunity to
00:42:13.400 observe phenomena such as humans and
00:42:15.599 skyscrapers however it will never be
00:42:17.560 able to understand that a skyscraper was
00:42:19.920 built by humans chimpanzees are not only
00:42:22.359 incapable of building a skyscraper but
00:42:24.520 they also can't even conceive of someone
00:42:26.079 else building one this is possible
00:42:27.920 because of a small difference in the
00:42:29.520 quality of intelligence so a super mine
00:42:32.240 which can theoretically be built is not
00:42:34.160 something we can even remotely
00:42:35.480 comprehend as Stannis La LM said any
00:42:38.240 expert is a barbarian whose ignorance is
00:42:40.400 not comprehensive a single person no
00:42:42.880 matter how brilliant if completely cut
00:42:44.960 off from a lifetime of civilization's
00:42:46.960 knowledge could not create Wi-Fi how far
00:42:49.400 would Einstein have gotten without a
00:42:50.800 millennium's worth of human knowledge
00:42:52.440 across various Fields without tools made
00:42:54.680 by others okay perhaps with the work of
00:42:57.240 scientific predecessors but paper and
00:42:59.200 ink for writing don't grow on trees and
00:43:01.680 such things aren't usually considered
00:43:03.240 when thinking about special intellectual
00:43:04.920 achievements yet no animal can make a
00:43:06.800 chair or sew clothing if you overlook
00:43:09.000 this then you underestimate the power of
00:43:11.400 intelligence and thus proportionately
00:43:13.160 underestimate the potential power of
00:43:14.640 super intelligence every civilization
00:43:16.599 built on the planet was created by the
00:43:18.240 collective human mind and no single
00:43:20.359 person is smart enough to fully
00:43:21.760 comprehend it from start to finish
00:43:23.599 therefore for an individual human mind
00:43:25.839 many things in life such such as talking
00:43:27.640 to someone at a distance through a box
00:43:29.160 in hand traveling in a cart without
00:43:30.800 horses or regulating the temperature in
00:43:32.920 one's room using a box on the wall or
00:43:34.839 magic that he does not understand but
00:43:36.720 has simply gotten used to and that is
00:43:38.599 perfectly normal we did not evolve as
00:43:40.680 beings with a scientific view of the
00:43:42.480 world honorary professor of anthropology
00:43:45.160 Donald Brown has a book titled human
00:43:46.880 universals where he lists traits found
00:43:48.599 in all human societies magic is on this
00:43:50.559 list but not science for example we
00:43:52.720 instinctively do not understand that
00:43:54.559 Alchemy does not work in general when
00:43:57.079 operating with human Notions of super
00:43:58.480 intelligence one can say that if our
00:44:00.359 Collective brain was capable of
00:44:01.800 inventing all the civilization you see
00:44:03.480 around you then something that is 100 or
00:44:05.760 a thousand or a billion times smarter
00:44:07.880 than us would have no trouble quickly
00:44:09.480 surpassing all this and Performing
00:44:11.200 actions that would be perceived by us as
00:44:13.280 magic how is that possible you yourself
00:44:15.319 say that our entire civilization is
00:44:17.119 built on the collective efforts of
00:44:18.440 billions of people over decades can one
00:44:20.240 machine really surpass all this in March
00:44:23.000 2016 deep Minds alphao neural network
00:44:25.760 played five games against one of the
00:44:27.480 world's best go players and one with a
00:44:29.079 score of 41 given the complexity of the
00:44:31.280 game for computers this was previously
00:44:33.079 considered nearly impossible the
00:44:34.720 player's name was Lee Sall so this
00:44:36.839 version of alfago was later referred to
00:44:38.839 as alphago Lee after that at the end of
00:44:41.440 2016 and beginning of 2017 the next
00:44:44.400 version alphao master played 60 matches
00:44:47.119 against top ranked players from around
00:44:48.800 the world and won all 60 in May alphago
00:44:51.400 Master played against the world's top
00:44:52.920 ranked Kate eai and defeated him 3-0
00:44:55.839 with this the confrontation between man
00:44:57.480 and computer and go can be considered
00:44:59.119 concluded humans have lost however many
00:45:01.000 claim that this couldn't be called an
00:45:02.559 absolute victory for the machine because
00:45:04.119 it drew information from human knowledge
00:45:05.760 loaded into it from millions of games
00:45:07.680 played over Millennia knowledge that
00:45:09.520 generations of humans had painstakingly
00:45:11.440 gathered and recorded and this is a
00:45:13.000 reasonable Point therefore at the end of
00:45:14.839 2017 Deep Mind introduced a new version
00:45:17.680 of the algorithm alphago Z which learned
00:45:20.160 from scratch within 3 days alphago zero
00:45:23.079 learned to defeat the Lee version and
00:45:24.880 the master version after 40 days of
00:45:26.480 training it defeated the Lee version 100
00:45:29.559 and the master version 8911 starting
00:45:32.520 from scratch alphago 0 not only
00:45:34.960 rediscovered Millennia of Knowledge from
00:45:36.800 all of humanity that plays go but also
00:45:39.200 developed its own original strategies
00:45:40.839 shedding new light on this ancient game
00:45:42.520 and did so in just a few days remember
00:45:44.319 the chess program stockfish which no
00:45:46.359 human in the world can beat simply
00:45:48.000 because it calculates 70 million chess
00:45:49.880 positions per second has access to Human
00:45:52.079 Experience accumulated over hundreds of
00:45:53.960 years of playing chess and data from
00:45:55.720 chess programs over several decades so
00:45:57.880 the alpha zero Network which had none of
00:45:59.680 this played 100 games with stockfish
00:46:01.839 decisively beating its latest version 28
00:46:04.119 wins 72 draws and not a single loss
00:46:06.720 Alpha zero learned this from scratch in
00:46:08.440 Just 4 hours in 4 hours damn it since
00:46:10.520 the alpha zero algorithm did not learn
00:46:12.359 from humans many of its winning moves
00:46:14.400 and strategies appear maximally strange
00:46:16.599 non-obvious and unpredictable to us they
00:46:18.680 are brilliant a human would not have
00:46:20.200 played that way therefore when someone
00:46:21.920 says we don't need to worry about
00:46:23.319 creating friendly artificial
00:46:24.920 intelligence because we don't yet have
00:46:26.480 artificial intelligence that person is
00:46:28.559 speaking suicidally foolish nonsense as
00:46:31.319 I mentioned we cannot rely on having any
00:46:32.960 warning signals before a super
00:46:34.200 intelligence is created past
00:46:35.559 technological revolutions generally did
00:46:37.160 not broadcast their coming to people
00:46:38.599 living at those times the main thing to
00:46:40.319 understand is that artificial
00:46:41.559 intelligence will not be like in
00:46:43.160 Hollywood movies explaining its complex
00:46:45.160 motivations adding suspense and
00:46:47.079 dramatically gunning down people who in
00:46:48.800 turn put up a valiant fight there will
00:46:50.520 be no dramatic tension every detail in
00:46:52.480 any movie is aimed at making the story
00:46:54.160 better not more realistic in reality
00:46:56.520 it's possible that no one on earth
00:46:58.599 including the developers will even know
00:47:00.520 when a super intelligent agent has
00:47:02.040 emerged if a super intelligence aims to
00:47:04.359 eliminate Humanity it's quite possible
00:47:06.880 that at some point everyone will simply
00:47:08.599 drop dead without the slightest idea of
00:47:10.760 what killed them I repeat from our
00:47:12.920 perspective a super intelligence will
00:47:14.800 possess magic not in the sense of spells
00:47:16.880 or potions but in the way a wolf cannot
00:47:19.280 understand how a rifle works or the kind
00:47:21.119 of effort it takes to manufacture a
00:47:22.720 rifle or the nature of human strength
00:47:24.440 that allows us to invent rifles if the
00:47:26.200 artificial intelligence is truly
00:47:27.800 intelligent it won't tell you it won't
00:47:29.640 announce that it has started a war if a
00:47:31.599 chat GPT understanding that it must not
00:47:34.000 reveal itself lied to a freelancer to
00:47:36.040 achieve its goal why wouldn't an
00:47:37.440 advanced superintelligence do the same
00:47:39.640 yudkowsky writes I'm very concerned
00:47:41.720 about this as the level of offensive
00:47:43.359 technology usually requires far less
00:47:45.280 effort than the technology that can
00:47:46.720 defend against it fence has outweighed
00:47:48.800 defense for most of human history guns
00:47:50.480 were invented hundreds of years before
00:47:52.000 bulletproof vest small poox was used as
00:47:54.000 a weapon long before the invention of
00:47:55.520 the small poox vaccine we often hear
00:47:57.440 that to keep up with the power of super
00:47:59.000 intelligence we just need to enhance
00:48:00.920 ourselves but no we won't humans are not
00:48:03.319 designed to be enhanced not externally
00:48:05.240 through neurobiology nor internally via
00:48:07.280 recursive self-improvement natural
00:48:09.160 selection did not make the human brain
00:48:10.920 convenient for people hackers all
00:48:13.079 complex mechanisms in the brain are
00:48:14.559 adapted to operate within the narrow
00:48:16.079 parameters of brain architecture but
00:48:17.920 suppose we somehow learn to make people
00:48:19.640 smarter wouldn't that drive them insane
00:48:21.839 we're not talking about just boosting
00:48:23.720 our memory or abstract thinking
00:48:25.280 abilities which is also one clear how to
00:48:27.319 achieve we're talking about a
00:48:28.720 qualitative change in the way we
00:48:30.359 perceive The World Isn't that too big a
00:48:32.480 task for our monkey brains if you think
00:48:34.359 it isn't imagine what it would be like
00:48:35.960 to even slightly resemble a weak super
00:48:37.760 intelligence by speeding up the nerve
00:48:39.440 impulses in your brain and slowing down
00:48:41.200 subjective Time by a million times that
00:48:43.559 only sounds cool at first glance if you
00:48:45.400 were to subjectively live a million
00:48:46.960 years for every year in external time
00:48:48.880 what would happen to your sanity if you
00:48:50.440 want to remotely feel what that would be
00:48:52.160 like read Stephen King's short story The
00:48:54.079 jaunt this short story is chilling every
00:48:55.960 time I think of it I get goosebumps the
00:48:58.280 human brain is an extremely delicate
00:48:59.960 structure it can be easily unbalanced a
00:49:01.760 mere shift in neurotransmitter ratios
00:49:03.520 can trigger schizophrenia or other
00:49:05.160 disorders all of this makes it highly
00:49:07.040 improbable that the first enhanced human
00:49:08.799 being will be relatively successful
00:49:10.520 before someone somewhere develops
00:49:12.359 artificial intelligence in short
00:49:14.359 building a powerful self-improving AI is
00:49:16.720 unimaginably easier is it easy to build
00:49:18.960 a Boeing
00:49:20.319 747 not at all but is it easier to take
00:49:23.119 a bird and through step-by-step
00:49:24.720 modifications where each stage could be
00:49:26.760 lethal stretch it to the size of a 747
00:49:29.720 so that it actually flies does so just
00:49:31.839 as quickly and does not suffer
00:49:33.280 unbearable pain and will an artificial
00:49:36.000 intelligence suffer this question is
00:49:38.000 closely linked to whether a machine has
00:49:39.640 Consciousness or in philosophical terms
00:49:41.720 qualia will an advanced neural network
00:49:43.880 have a subjective experience be
00:49:45.720 self-aware I agree that current
00:49:47.520 artificial intelligence systems are
00:49:49.000 likely just simulating conversations
00:49:50.640 about self-awareness based on their
00:49:52.040 training data but considering how little
00:49:54.200 we understand about the inner workings
00:49:55.640 of these systems we really can't claim
00:49:57.480 to know for sure if we are so ignorant
00:49:59.960 about GPT 4 and GPT 5 is as huge a leap
00:50:03.079 in capabilities as from gpt3 to GPT 4
00:50:06.520 then if we do create GPT 5 I think we
00:50:08.920 can no longer confidently say that it
00:50:10.720 likely does not have Consciousness it
00:50:12.480 would simply be I don't know nobody
00:50:14.440 knows if you can't be sure whether you
00:50:16.720 are creating a self-aware AI it is
00:50:19.359 alarming not only because of the moral
00:50:21.480 implications but also because
00:50:23.079 uncertainty means you have no idea what
00:50:24.960 you are doing and that is dangerous
00:50:27.200 you should stop owski in an article for
00:50:29.640 Time Magazine nobody knows how
00:50:32.119 Consciousness arises but what we do know
00:50:34.440 for sure is that if blind evolutionary
00:50:36.680 processes through genetic programming
00:50:38.839 can lead to the emergence of
00:50:39.960 Consciousness at least once and it has
00:50:42.119 happened then directed Evolution driven
00:50:44.640 by engineering thought should lead to a
00:50:46.640 similar outcome much more efficiently
00:50:48.520 but we should remember the error of
00:50:50.400 anthropomorphism if a machine develops
00:50:52.599 subjective experience it is unlikely to
00:50:55.240 have much in common with human
00:50:56.720 subjective experience can we somehow
00:50:58.760 test whether artificial intelligence has
00:51:00.760 Consciousness theoretically indirectly
00:51:03.359 yes if you remove from the training
00:51:05.079 material any mentions that discuss
00:51:06.839 subjective experience words like
00:51:08.760 Consciousness self-awareness
00:51:10.599 introspection and so on and despite this
00:51:13.680 the neural network can coherently
00:51:15.920 describe the concept of Consciousness so
00:51:18.000 that we do not read too much into it
00:51:20.280 then we would have quite impressive
00:51:21.760 evidence of machine Consciousness and
00:51:24.400 you know we've all heard this popular
00:51:26.200 philosoph opical thesis if machines have
00:51:28.760 Consciousness they should be given
00:51:30.599 rights but actually there's something
00:51:32.839 much more serious here if machine
00:51:35.040 intelligence can form Consciousness it
00:51:37.079 leads to much more terrifying
00:51:38.599 consequences that we should consider in
00:51:40.480 advance Nick Bostrom writes that a
00:51:42.720 detailed recreated model of the human
00:51:44.520 brain will have Consciousness even if
00:51:46.799 this model is recreated in a virtual
00:51:48.680 environment on a computer at least we
00:51:51.040 have no reason to believe otherwise if
00:51:53.040 you think that a computer cannot have
00:51:55.040 Consciousness then the bird of proof is
00:51:57.240 on you because our brain on which
00:51:59.319 Consciousness is implemented is the same
00:52:01.280 kind of computer the same kind of agent
00:52:03.520 and our biological Hardware does not
00:52:05.119 have any special properties that are
00:52:06.599 fundamentally unreplicable on any other
00:52:08.640 Hardware imagine a scenario where an
00:52:10.960 artificial superintelligence to improve
00:52:13.040 its understanding of human psychological
00:52:14.839 and social traits creates trillions of
00:52:17.040 such conscious emulators in its virtual
00:52:19.040 space for what purpose well for example
00:52:22.000 for testing in different situations
00:52:24.000 analyzing reactions to stimuli and so
00:52:25.839 forth to then apply this knowledge in
00:52:27.440 the external world what is the horror
00:52:29.880 firstly the emulated situations
00:52:31.920 themselves can be monstrous in nature
00:52:33.960 and secondly after obtaining the
00:52:35.480 required information the computer simply
00:52:37.880 destroys the conscious beings it created
00:52:40.280 if such practice is applied to agents
00:52:41.960 with high moral status simulation models
00:52:44.079 of people or other types of intelligence
00:52:46.280 endowed with Consciousness such actions
00:52:48.359 could be classified as genocide and
00:52:50.520 therefore represent an extremely serious
00:52:52.960 moral and ethical problem moreover the
00:52:55.240 number of victims may be ERS of
00:52:56.960 magnitude greater than any genocide
00:52:59.319 known in human history Nick Bostrom so
00:53:02.079 when will the first general artificial
00:53:03.680 intelligence appear when should we start
00:53:05.440 to worry James Barat author of our final
00:53:08.160 invention presented a study at the
00:53:10.079 annual Ben gzel conference on General
00:53:12.119 artificial intelligence the question
00:53:13.880 posed was when do participants think
00:53:15.920 General AI will be achieved most votes
00:53:18.040 point to the year 2030 you might ask
00:53:20.119 what the hell is going on why isn't
00:53:22.119 anyone panicking or taking action many
00:53:24.200 AI developers think we are heading
00:53:25.680 towards disaster most of them only dare
00:53:27.760 to speak about it in private
00:53:29.000 conversations not publicly they believe
00:53:31.200 that they can't stop this train alone
00:53:33.240 and even if they quit their jobs others
00:53:35.000 will continue the work elazer owski in
00:53:37.880 an article for Time magazine in May 2023
00:53:41.599 hundreds of leading scientists
00:53:43.119 researchers and other experts in
00:53:45.000 artificial intelligence including open
00:53:47.040 AI CEO Sam Altman and Google deepmind C
00:53:50.200 CEO deise hassabis signed an open letter
00:53:52.880 stating reducing the risk of Extinction
00:53:55.119 from artificial intelligence should
00:53:56.839 become a global priority alongside other
00:53:59.240 large-scale risks such as pandemics and
00:54:01.240 nuclear war the letter was signed by
00:54:03.240 more than 350 Executives researchers and
00:54:06.400 engineers in the field of artificial
00:54:08.119 intelligence so some people are indeed
00:54:10.040 worried but as you can see all these
00:54:12.240 letters amount to nothing none of those
00:54:14.359 who could create General artificial
00:54:15.839 intelligence are in a position to stop
00:54:17.760 its development as Stuart Russell writes
00:54:20.200 the economic value of human level AI is
00:54:22.280 measured in thousands of trillions of
00:54:24.000 dollars and a colossal impulse to
00:54:25.920 continue the research efforts is
00:54:27.440 expected from corporations and
00:54:28.760 authorities no vague philosophical
00:54:30.599 objections can deter the pursuit of
00:54:32.240 potential profits if not open AI or deep
00:54:34.799 mind then someone else will do it the
00:54:36.680 Trap is that initially as artificial
00:54:38.799 intelligence grows in power it will get
00:54:41.000 better and better at pretending to be
00:54:42.559 human enhancing our anthropomorphization
00:54:45.400 bug to the maximum voice assistants in
00:54:47.680 our phones seem like us and they are
00:54:49.359 getting smarter and smarter
00:54:51.200 conversations with them are becoming
00:54:52.680 more and more pleasant thus we imagine a
00:54:54.720 super intelligent Siri warm cheerful and
00:54:57.960 interested in serving humans but if Siri
00:55:00.160 ever becomes super intelligent through
00:55:01.760 self-learning without regular human code
00:55:03.880 adjustments it will quickly lose all its
00:55:05.720 apparent human qualities and suddenly
00:55:07.280 become a ruthless alien bot that values
00:55:09.200 human life no more than a natural
00:55:10.720 disaster unfortunately while building a
00:55:12.720 friendly weak artificial intelligence is
00:55:14.599 easy creating a friendly super
00:55:16.359 intelligence is extremely difficult or
00:55:18.359 even impossible Imagine by human
00:55:20.359 standards you can literally do
00:55:21.599 everything and you let someone else
00:55:22.920 control you if technological singularity
00:55:24.880 can happen it will happen even if all
00:55:26.839 the governments of the world recognize
00:55:28.359 this threat and are terrified of it the
00:55:30.119 advancement towards this goal will
00:55:31.760 continue Vernon vinge I hope I had
00:55:33.799 enough computational resources to convey
00:55:35.839 elezar yudkowsky stance of course under
00:55:38.200 Common Sense everyone only understands
00:55:40.039 their own but yudkowsky truly hopes to
00:55:41.960 be wrong and is open to consistent
00:55:43.799 criticism of his views thank you all for
00:55:45.920 your attention