# Lecture 16.1: Introduction to Artificial Intelligence (AI)

[Recording](https://www.youtube.com/watch?v=nrIKNdeQ9m8)

Automatic transcript; technical terms and equations may contain recognition errors.

00:00:00 So today we're gonna move on to the third module, Deep Learning.

00:00:06 And in its first lecture of the module,

00:00:10 we'd like to talk about AI and its impact everywhere, including health.

00:00:16 So that's the goal today.

00:00:19 That's where we are.

00:00:20 So again, we have already studied a classical problem and

00:00:26 its classical approach.

00:00:29 And we have seen through these examples

00:00:34 what optimization and machine learning problems are like.

00:00:39 And we have seen already as well, even in the machine learning cases,

00:00:44 what's optimization formulation and what's optimization algorithms.

00:00:49 So now I think it's time to think again.

00:00:53 At the very beginning, I suggested us to think from the data perspectives

00:01:00 for this kind of problems, right?

00:01:03 So we have protein sequence, structure, and other kind of data,

00:01:08 like protein-protein interaction networks, protein expressions, and so on.

00:01:13 And we have functions as well.

00:01:15 So there are different kind of data of different modalities.

00:01:20 And we are very interested in learning the distribution

00:01:26 within each modality, as well as the relationships across modalities.

00:01:33 So that's another perspective.

00:01:36 So with this, we'll talk about deep learning more and more,

00:01:40 including these contents.

00:01:42 I actually have sent a short list of eight papers

00:01:47 covering CNN, RNN, language models, generative models

00:01:56 in the list.

00:01:57 And I was asking for your preference, one, two, three,

00:02:00 so that we can find the optimal assignment of your journal club presentation.

00:02:07 So we expect to start the journal club presentations after the spring break,

00:02:11 which is next week.

00:02:15 As a reminder, the homework two is due today.

00:02:18 So if you need extension, please contact me early.

00:02:22 And then we originally had the proposal due next Thursday,

00:02:29 but it's a spring break, so we can extend it for three days,

00:02:33 but not too much longer to the Monday after next week.

00:02:39 OK?

00:02:40 We will also assign a homework three this week, which is very short.

00:02:45 It probably only takes half an hour for some of you,

00:02:48 but it could take an hour for some others.

00:02:50 It's a machine learning programming homework.

00:02:54 We can, again, set it to be due, like, the week after next week,

00:02:59 like a late the week after next week.

00:03:02 We'll decide.

00:03:03 And then, again, if you need extensions,

00:03:05 but this one is even shorter than homework two, OK?

00:03:09 So that would be all of our homeworks.

00:03:12 And I expect to return the graded homework one during or right after the spring break.

00:03:17 So that's a great question.

00:03:20 Any questions about the logistics?

00:03:24 OK?

00:03:24 Please read the short list and cast your vote, one, two, three,

00:03:31 in that Excel sheet by the end of this week before the spring break, OK?

00:03:36 And as I said, I will solve an optimum assignment problem

00:03:41 by formulating it into integer linear programming,

00:03:46 very similar to the first problem in your homework two, OK?

00:03:51 I'm actually going to use the one, two, three matrix you give me

00:03:55 and solve that integer linear programming

00:03:58 and find the best assignment for the journal club, OK?

00:04:02 Let's see how that works.

00:04:04 When you give your one, two, three, don't look at other columns.

00:04:08 Let's make it as independent as possible, OK?

00:04:12 All right.

00:04:13 So again, this was already announced in the announcement last night.

00:04:21 So and after you're giving this preference,

00:04:25 we'll give an assignment and give the date, OK?

00:04:29 Please send me the slides no later than the noon on the day of the presentation, OK?

00:04:39 Oh, actually, on the day before the presentation,

00:04:41 by the noon before the day of presentation, OK?

00:04:45 So you can make updates between sending me and presenting the slides.

00:04:52 But I'd like to load your slides at the beginning of the lectures to save time.

00:04:58 It takes about 20 minutes, and we'll also ask people to have questions.

00:05:04 So everybody is expected to ask questions, OK?

00:05:09 All right.

00:05:09 So what are we going to talk about?

00:05:11 So today we're going to talk about AI.

00:05:16 This is not a new word.

00:05:21 It's an old word.

00:05:23 Well, relatively new.

00:05:24 Well, we'll see when it was coined.

00:05:26 But the concept is older than the name.

00:05:31 We'll talk about its background and what it was meant for.

00:05:37 And what are the recent breakthroughs.

00:05:40 And then we'll see how the AI's impacts have outreached to many fields, including healthcare, OK?

00:05:48 And in the end, I want you to have this data view of our problems.

00:05:55 So there's artificial intelligence.

00:05:57 There's biological intelligence at all scales.

00:06:00 We're now just looking at molecules, but there are biological intelligence at all scales, OK?

00:06:05 So toward AIBI, that's, you know, my philosophy.

00:06:12 And in the end, if time allows, we'll give a first take on neural networks, OK?

00:06:20 So let's get started.

00:06:23 So what is artificial intelligence?

00:06:25 Before proceeding, what is artificial intelligence to you?

00:06:33 It doesn't have to be an abstract concept.

00:06:37 It could be a concrete example or anything.

00:06:40 Can anyone?

00:06:41 Actually, we only have a few people here, so I'm asking each of you, give me one answer.

00:06:47 OK, so let me take a random order.

00:06:50 Nami?

00:06:51 Any tool or machine that can do at the accuracy performance level of human beings?

00:07:01 So any what machine?

00:07:03 Any tool or machine.

00:07:04 Any tool or machine that can perform some task close to or at the same level of accuracy as humans?

00:07:12 Is that right?

00:07:14 Wow, OK.

00:07:16 OK, thank you.

00:07:17 I mean, good.

00:07:18 That's a good prior.

00:07:19 OK, great.

00:07:21 Let's see.

00:07:22 Hey, Ming.

00:07:23 What is AI to you?

00:07:27 What is AI to you?

00:07:29 I was thinking more in the line of any sort of algorithms that humans created that solves problems.

00:07:39 Any kind of algorithms that humans created and what?

00:07:43 Old, like vacuum tubes, like computer data, could be considered artificial intelligence since they're artificial.

00:07:53 So you're saying that any, not just algorithms, but also in the tools or physical systems that create it.

00:08:00 It could be like too much broader if you, in my sense.

00:08:05 So that's where artificial is from, but what about intelligence?

00:08:09 Well, then being able to.

00:08:12 Being able to.

00:08:13 Calculate stuff.

00:08:14 Calculate stuff.

00:08:15 In sense.

00:08:16 Intelligence.

00:08:17 Good, good, good, good.

00:08:18 OK, you're all hitting something, some key elements.

00:08:24 That's great.

00:08:24 OK, let's, Doyoung?

00:08:28 What's AI to you?

00:08:30 To me, any machine that, any machine that will make human give less effort to get an output from some input?

00:08:44 Uh-huh.

00:08:45 Yeah.

00:08:47 Any, any machines that can, can, can, you know, liberate human labor and, you know, perform the, some task of getting some output from the input.

00:08:59 Is that right?

00:08:59 Yeah.

00:09:00 OK, great.

00:09:01 Thank you all.

00:09:01 And as I said, all of you have some key elements and, and I really like it.

00:09:08 So the background.

00:09:11 As we know, AI is an old concept, a relatively new world.

00:09:19 It's a coined word.

00:09:22 Born in 1950s, but the concept is earlier than 1950s.

00:09:27 OK, we'll talk about how it's born as a word.

00:09:32 And then, AI is a very broad field.

00:09:38 It can, you know, includes a lot of research topics and machine learning is just one of them.

00:09:45 And it began to flourish from the 1980s.

00:09:49 OK, you know, look, we, we, we built computers, you know, way in 40s or 50s already.

00:09:56 Right? But the machine learning began to flourish in, in 1980s.

00:10:01 And deep learning, as you know, is a subset of machine learning.

00:10:06 It began to flourish thanks to the breakthrough of the CNN in ImageNet competition. Right?

00:10:20 And we'll, we'll, we'll review that as well as when we talk about CNN.

00:10:25 So in other words, AI includes machine learning and machine learning includes deep learning.

00:10:33 So it's increasingly smaller and smaller subset.

00:10:36 But now, as AI is getting more and more popular as a buzzword, very often when people are talking about AI,

00:10:47 they are actually just talking about deep learning.

00:10:49 OK, so that's, that's the, the, the situation.

00:10:54 So, so how was the field of AI was born?

00:11:00 Right? So, you know, let's, let's look at this AI, right?

00:11:04 So, so this artificial intelligence, as I, you know, mentioned earlier,

00:11:11 it already exists to some extent before the world was created.

00:11:18 You know, remember the first lecture, we mentioned Norbert Wiener,

00:11:24 and the book called Cybernetics, the control and the communication in animals and in machines.

00:11:32 Right?

00:11:32 So that to some extent is, you know, related to artificial intelligence, right?

00:11:38 Back then, remember, it was 1950s after the World War II, there are a lot of, you know,

00:11:43 automata theory and other kind of research already going.

00:11:46 So, so it, as a concept to some level, is already existing.

00:11:52 OK, we'll, we'll talk about Turing for sure.

00:11:54 And then, and again, emphasize that.

00:11:58 OK, but how was this word created?

00:12:02 So in, in 1950s, there is a very young assistant professor at Dartmouth College,

00:12:09 John McCarthy, right?

00:12:11 So, so he actually approached the Rockefeller Foundation and wanted to submit a proposal.

00:12:21 OK, back then, he and the Cloud Shannon together talked with the director of biological and medical

00:12:28 research at the Rockefeller Foundation.

00:12:31 And they, you know, proposed the idea of having a brainstorming seminar at Dartmouth College in that

00:12:40 summer of 1950s.

00:12:41 OK?

00:12:43 And eventually, it was meant for us in a small group of 10 people, eventually a little bit above 10

00:12:49 people attended.

00:12:51 And the proposal was by John McCarthy and then, you know, Cloud Shannon, we all know, information theory,

00:13:00 right?

00:13:01 So, do you know Marvin Minsky?

00:13:08 The founder of the AI lab at MIT and what he's famous for?

00:13:16 A lot of things.

00:13:21 Among many,

00:13:22 the first or the second winter of the AI was largely thanks to his breakthrough research.

00:13:33 OK, we'll talk more about it.

00:13:36 And then, Nathaniel Rochester, you don't know him or know about him, do you?

00:13:42 The chief architect of the IBM 701.

00:13:51 OK, the first scientific computer.

00:13:55 OK, so these are the people who wrote this proposal.

00:14:00 OK, let's see what they were talking about.

00:14:03 OK, so the study is to proceed on the basis of the conjecture that

00:14:09 every aspect of learning or any other feature of intelligence can, in principle, be so precisely

00:14:18 described that a machine can be made to simulate it.

00:14:24 OK, so that's assumption.

00:14:25 OK, so the every aspect of learning or the key feature of the intelligence can be, in principle,

00:14:32 precisely described so that a machine can be made to simulate it.

00:14:37 OK, so something that you covered.

00:14:39 So, in other words, this concept, you know, the field is about how to make machines use language,

00:14:47 form abstractions and concepts, solve kinds of problems now reserved for humans and improve themselves.

00:14:54 OK, so you all also covered something like this.

00:14:56 OK.

00:14:57 OK, and so in other words, it actually covers all the remaining reasoning, problem solving, knowledge representation,

00:15:06 planning, learning, natural language processing, perception, motion and manipulation, society intelligence and general intelligence and so on.

00:15:16 So in other words, learning is part of it and deep learning is a subset of machine learning.

00:15:23 Right. So now a lot of people are using the buzzword AI, but actually what they refer to is a subset of AI, which is deep learning.

00:15:34 Right.

00:15:36 And back then, John McCarthy thought that he would like to create a new word, artificial intelligence,

00:15:42 despite the, you know, the other kind of related research, partly because he want to distant from those fields.

00:15:49 Right. Because those fields are already established with leaders.

00:15:55 Right. They have their own focus and some of them are, you know, hard to argue with.

00:16:01 So they thought, well, let's, you know, focus on what we want to understand and create a new field and call it AI.

00:16:10 So it's very important how you position yourself.

00:16:13 OK, so they made a right choice, but don't forget that at the beginning, these people.

00:16:19 You know, it's a dream team.

00:16:20 OK.

00:16:21 All right. So that's the birth of the AI as a word.

00:16:27 OK.

00:16:29 But as you know that AI research exists before the world was created.

00:16:36 Right. So one of the most famous scientists and we're, you know, familiar with his name is Alan Turing.

00:16:44 OK.

00:16:45 So as you see, you know, at the early days, you can wonder how human intelligence can be described and how machines can simulate.

00:16:57 But how do you approach such questions is very important.

00:17:02 Look, you can ask a very fundamental and important question.

00:17:09 But more importantly, well, that's half of the success.

00:17:14 But the remaining half of success is equally important is how do you approach such questions?

00:17:22 How do you plan milestones to approach that question step by step?

00:17:27 Right.

00:17:28 You cannot ask a question that cannot be easily answered in the current technical world.

00:17:36 Right. So so what what Alan Turing's contribution is changing the question, very general question,

00:17:46 can machines think?

00:17:49 OK.

00:17:49 As I said, you know, how do we precisely describe human intelligence and let machines assimilate it?

00:17:55 So this question is unfortunately very hard to answer.

00:18:00 And it's hard to you would say yes or no or what?

00:18:03 I mean, it's not very tangible.

00:18:06 You know, it doesn't really give you the right incentives in the specific research activities.

00:18:11 Right. It's hard to quantify and it's really hard to even know what's the impact of the answer.

00:18:17 Do you see what I mean?

00:18:18 If I say yes, what do you do?

00:18:20 If I say no, what do you do?

00:18:21 Right.

00:18:22 So he actually smartly translated the question of can machines think to can machines do what we

00:18:30 as thinking entities can do?

00:18:33 OK.

00:18:34 So this is a very important turning point.

00:18:39 OK.

00:18:40 It's a critical moment.

00:18:41 That's the famous Turing test for machine intelligence.

00:18:45 OK.

00:18:46 So when I heard Nami's answer, I thought, wow, that sounds like some element of Turing test.

00:18:54 Right.

00:18:54 So I was happy about that.

00:18:56 So here what you have is like, you know, you have a top view, right, of some cubicles.

00:19:02 Right.

00:19:02 They cannot see each other.

00:19:04 Right.

00:19:04 Is this a top to bottom view?

00:19:06 There is a computer machine and there is a human.

00:19:09 Right.

00:19:09 They are both answering the same exam.

00:19:13 OK.

00:19:13 So how you design the exam is another question.

00:19:16 And there is a proctor.

00:19:17 Right.

00:19:18 The proctor cannot see who, you know, he doesn't know whether the left cubicle is a computer

00:19:23 or a right cubicle is a human or computer.

00:19:26 He just look at the two answers and then decide whether, you know, he can differentiate the two

00:19:33 based on their answers.

00:19:34 Right.

00:19:35 So this is the Turing test.

00:19:38 OK.

00:19:38 If you think about this, as I said, it's a critical point of, you know, making this study

00:19:46 of artificial intelligence much more tangible and much more meaningful.

00:19:51 OK.

00:19:51 So as you will see later on, if you wonder why AI research turned out the way they were in 1980s

00:19:59 or early 2000s, you will understand that it's Turing test, this philosophy of Turing test behind it.

00:20:07 OK.

00:20:08 So I will remind you that again.

00:20:10 So let's talk about the AI and the recent breakthroughs.

00:20:15 OK.

00:20:17 I will talk the following.

00:20:19 The first revolution is man versus machine.

00:20:23 OK.

00:20:23 So use games as test beds.

00:20:26 OK.

00:20:27 So you may wonder why we want to use game as test beds.

00:20:33 Because that's the perfect setting for Turing test, isn't it?

00:20:38 Because you have the man versus machine on the same common platform.

00:20:44 And these games at common platforms are already with established rules.

00:20:50 Right.

00:20:50 Right.

00:20:51 So you can unbiasedly just, you know, look at their performances and judge.

00:20:55 So that's why in the early days, well, not so early, the recent AI revolution, they started with the games.

00:21:02 OK.

00:21:03 And then the next is the natural language processing.

00:21:08 It's in the text and the speech data.

00:21:12 And the next one is a computer vision.

00:21:14 That's the image data, including how you analyze the image data and how you generate image data.

00:21:20 OK.

00:21:21 So and there's more.

00:21:22 OK.

00:21:23 But let's focus on these three first.

00:21:26 First, man versus machine.

00:21:28 All right.

00:21:29 As I said, this is a perfect test bed for Turing test.

00:21:34 OK.

00:21:35 So the reason that, you know, early AI research, right, focus on the games, not for the sake of the games themselves.

00:21:44 Of course, you know, it gains popularity when you see that Deep Blue from IBM won the chess game, right, against a world champion.

00:21:57 Right.

00:21:58 So maybe two out of three or something like that.

00:22:00 But it was as early as 1997.

00:22:04 OK.

00:22:05 So still, after Deep Blue, people thought that, yes, that's amazing.

00:22:16 But the complexity of a chess game is still, you know, OK, if you can consider some other games like Go.

00:22:27 OK, this ancient Asian game, right, whose complexity is much higher.

00:22:35 I will tell you how much higher later.

00:22:38 So people thought that, well, let's see.

00:22:40 I mean, this is the beginning.

00:22:42 But it will probably take another 20 years, OK, or 30 years, 20 years at least, to see a computer program beating a world gold champion.

00:22:56 OK.

00:22:56 OK.

00:22:57 Now, we'll see how long it took.

00:22:58 And also, I want to remind you that this is not the first chess algorithm or computer program.

00:23:08 OK.

00:23:09 Before that, I will show you the whole history.

00:23:11 Before that, there were already a lot of other programs.

00:23:14 OK.

00:23:14 But this is the first time that we see it has reached the level of potentially matching or beating a human world champion.

00:23:25 That's a quantum difference.

00:23:30 So how long did it take from 1996 to 2015, right?

00:23:34 So a little less than 20 years, right?

00:23:38 So this is year 2015, AlphaGo versus the world champion, Li, right?

00:23:47 So if you play Go or you follow the Go news, you would know that back then he was the Michael Jordan of basketball, right?

00:23:58 So and I want to remind you that the complexity of Go is a gu-gao term more than that of chess.

00:24:08 It's not my typo.

00:24:09 It's gu-gao.

00:24:09 So what's the order of magnitude for Google?

00:24:17 10 to what?

00:24:22 You want to guess?

00:24:28 You want to guess?

00:24:31 10 to 1?

00:24:33 10 to 10?

00:24:33 10 to 100.

00:24:39 So the complexity of Go is 10 to 100 than that of chess.

00:24:44 So people thought that this day will become a little, you know, it will come eventually, but it's going to come a little later.

00:24:51 And this was the game, right?

00:24:54 But as I said that back then AlphaGo, right?

00:24:58 So it was based on the Monte Carlo tree search, right?

00:25:03 And it was trained on data created by human, right?

00:25:06 So, you know, so that was trained on data created by human.

00:25:10 Humans playing humans, right?

00:25:12 And then they further evolved, right?

00:25:15 So they basically, you know, let now they train over time using large quantities of human knowledge and supervision.

00:25:23 A new system taught itself to master the program.

00:25:26 So, you know, you know, it's like a baby learning the new knowledge, right?

00:25:31 So it knows that in this world, right?

00:25:35 What is the rule?

00:25:36 So it keeps trying to play the game, right?

00:25:40 And then gradually grasp and reinforce the knowledge.

00:25:44 And then you see that, you know, it eventually start from the, you know, naive player and then it actually can learn, right?

00:25:55 It can learn from playing with itself and then gradually improve in 40 days.

00:26:02 So that's another level of AlphaGo called AlphaGo Zero, okay?

00:26:08 Okay.

00:26:09 And again, this was just the epitome of man versus machine in games.

00:26:19 Because, you know, as I said, it's a perfect, it's a perfect test bed for Turing tests.

00:26:26 So there are all kinds of AI products ever since the 1950s, okay?

00:26:32 So in 1956, IBM checkers AI first demonstrated, you know, so that was 1950s, okay?

00:26:39 So I know IBM may not be the first name you think of right now when you think of AI, but, you know, in more than five or six decades, IBM has been leading the AI revolution, among others, okay?

00:26:54 So, okay, the first chess AI, right, fully functional was in 1958, right?

00:27:01 And, yeah, the checkers AI actually wins in 1962.

00:27:07 Those were, you know, the checkers, right, relatively simple, right?

00:27:10 And the first world computer chess champion, you know, back then they already have the computer chess champion.

00:27:19 That was in 1974, Kaiser, right?

00:27:23 So the CNN as a concept was introduced in 1989 and back propagation in 1989, 86, right?

00:27:30 So we'll talk about them when we talk about deep learning.

00:27:34 In the meantime, we see technology-wise, reinforcement learning and neural network-based back common AI, you know,

00:27:43 as research has been shown in 1990s, early 1990s, right?

00:27:49 So the Monte Carlo goal was in 1993 and the Monte Carlo tree search goal in 2026, 2006, okay?

00:27:57 And then got improved by deep mines, AlphaGo and AlphaGo Zero, right?

00:28:05 So this Monte Carlo tree search-based goal actually, you know, already started in 2006, okay?

00:28:16 But it still took some years until it actually reaches some level and eventually beat a human world champion, right?

00:28:26 That's, you know, that took more than 10 years from 2006, okay?

00:28:32 And so there's a whole history behind it.

00:28:37 If you are interested, you can look at this blog, okay, or editorial.

00:28:45 So after man versus machine, let's talk about natural language processing, okay?

00:28:51 So it's actually so popular right now, but first I still want to bring back IBM, okay?

00:29:00 So in 2011, right, you must have heard of this, 2007, not 11, sorry, that's 11 years ago.

00:29:08 How old were you?

00:29:09 You were middle school students, right?

00:29:12 Did you hear about IBM Watson wins this game called Jeopardy?

00:29:21 Right?

00:29:21 You must have heard about something like that.

00:29:24 And that was in 2011.

00:29:26 So in this kind of game, you have to process the natural languages, okay?

00:29:31 So it's one of the early breakthroughs.

00:29:36 And then ever since then, natural language processing has been seen in so many applications.

00:29:43 For instance, virtual assistants, right?

00:29:45 So the first chatbot actually was already invented in 1966, okay?

00:29:52 But now I think it's much easier and natural for you to talk with a chatbot, right?

00:29:59 Like if you're talking with GPT, right, version 3, right, maybe it feels so, you know,

00:30:06 you can do your own Turing test, okay?

00:30:08 So, okay.

00:30:11 Then, you know, for instance, you know, there are a lot of email assistants, right?

00:30:16 You know, when you send an email, you CC those email addresses, and those are actually, you know,

00:30:22 email assistants that can directly, based on the content of your email, you know, assign this task

00:30:30 into your Google task, or put this on your Google calendar, or send this to your Slack bots, right?

00:30:38 So it's like a virtual assistant, exactly like, you know, someone, you know, it's like you hire a secretary, right?

00:30:47 And this processing of natural languages don't have to just go through the form of text, it can go through the voices, right?

00:30:55 So in 2011, Siri was born, right?

00:30:59 That was Facebook, and then Cortana, right?

00:31:03 So that's Microsoft 2014, Alexa, like, Amazon, right?

00:31:11 2014, Google Assistant, 2016, actually Google Now was even earlier, 2012, okay?

00:31:19 So this natural language processing is now in our day life, right?

00:31:25 You know, you can talk to Alexa, right?

00:31:28 And play this music, order that thing, right?

00:31:33 Another thing you may realize is that now

00:31:40 a lot of the translations are, you know, getting more and more real-time and precise.

00:31:46 That's also largely thanks to natural language processing, okay?

00:31:50 It's not only about processing the information from the text and the voice and do actions.

00:31:56 It's also about translating across different languages, okay?

00:32:00 So Google's neural machine translation was in 2016, okay?

00:32:08 I want to remind you that the early days of Google's translation was not based on,

00:32:13 that was not really that much based on, you know, what you imagined, okay?

00:32:21 At the early days, it was actually more based on the search, okay?

00:32:26 But now it's really building more intelligence into it.

00:32:33 And, you know, now you probably sometimes even get annoyed that when you are typing your email

00:32:39 or your text message, sometimes it auto-fills or complete the sentence for you, right?

00:32:47 And sometimes it could be a huge mistake or joke sometimes, right?

00:32:52 So the last revolution after games and natural language processing is in image data, okay?

00:33:00 So the image data, again, is getting more and more widely accessible, right?

00:33:09 Thanks to the digital cameras, right?

00:33:11 And the storage, you know, so as data get, excuse me, as data get more and more abundant,

00:33:18 then these technological improvement get more possible, okay?

00:33:23 So the data storage retrieval, all this, you know, data creation, collection, storage, retrieval,

00:33:33 and corresponding hardware, once these are mature enough, then, you know,

00:33:40 the software level artificial intelligence improvement or advancement become possible, right?

00:33:46 So I have to talk about ImageNet, right?

00:33:49 So as I said, for AI, you know, you are asking about the question whether the human intelligence

00:33:59 can be precisely described in a way that a computer can simulate it, right?

00:34:08 And you know that this problem is much more approachable using the Turing test.

00:34:14 In some cases, like games, that Turing test platform is very, you know, ready to accept available.

00:34:23 That's why early, you know, AI research was so focused, not so focused, but was massively using

00:34:30 games as a test bet, right?

00:34:32 But in some other cases, like this, you know, for instance, this image tasks, right?

00:34:37 If you want to tell or annotate an object, right, on this image,

00:34:43 or you want to segment the image, like annotate which pixels or which blocks of the images

00:34:51 corresponds to the crosswalks or the traffic lights, right?

00:34:55 So that kind of thing is, again, at the beginning is limited by the data and the rules, especially the

00:35:03 rules. In other words, the label. There's not a lot of labeled data, okay?

00:35:09 So maybe after you get a lot of data after the collection and the storage and retrieval hardware,

00:35:16 but now you're missing the labeled data, okay? So that's why Fefe Li started this ImageNet, right,

00:35:24 which is a great initiative, right? So it's, again, it's about the platform that makes the visual

00:35:32 intelligence possible, okay? It's like that Turing test bet, okay? So this makes it possible.

00:35:40 And now actually, you know, this labeling efforts are largely accelerated, right?

00:35:48 Thanks to what? The crowd wisdom, right? Amazon Turk, right? You can, you know, you can,

00:35:55 you know, hire people, right, with relatively cheap budget so that they can just label images for you,

00:36:03 and then sometimes even for free, right? You must have done it before. Have you heard of CAPTCHA?

00:36:11 Have you heard of re-CAPTCHA? Sometimes you are suspected that you're a robot, then it asks you,

00:36:17 please label all the images that have a bike or a pedestrian, right? So as you were testing yourself

00:36:26 to be a human, you are also donating your labeling efforts, right, to this database,

00:36:32 whether you realize it or not, right? So anyway, ImageNet then, you know, grows into a competition,

00:36:40 right? So any field, if it wants to advance, it needs standardized and, you know, abundant data and

00:36:51 assessment, okay? So ImageNet is making that possible. And then over years, people use ImageNet as a test

00:36:59 bed for visual intelligence and testing their algorithms, and they have a, you know, lump sum

00:37:06 performance metric, which is the slower the better, right? And the average human can reach 5.1, okay?

00:37:12 But over the years, they see the scores gradually decreasing, but looks like it's saturating, right?

00:37:21 And, but then one year, that is 2012, okay? So before 2012, people see that these methods are getting

00:37:32 more and more sophisticated focusing on a lot of human manual and, you know, feature tuning and all

00:37:40 other kind of things. Now, all of a sudden, in 2012, this performance metric drastically improved,

00:37:49 and people were asking, what just happened? So this is ImageNet in 2012, and eight years later,

00:37:58 the same thing happened in protein structure prediction. So there is a platform called CASP,

00:38:07 Critical Assessment of Structure Prediction, right? So it's been probably as old as you, like 20,

00:38:14 20 years as a concept. And in practice, maybe 16 years, 15, 16. That's the same thing in that year,

00:38:25 2020, people, you know, wonder what happened. Actually, even before the CASP competition

00:38:33 was finished, during the competition, because of the targets, the protein structure prediction targets,

00:38:38 were gradually rolled out. So the organizers and assessors can already see the performance of the

00:38:44 earlier submissions. They already noticed that there was one team that keeps submitting high-quality

00:38:52 predictions that just way outperformed others. They were even wondering, is this guy cheating?

00:39:00 So that was what we later know as AlphaFold2. Okay? So that's what I call it, the ImageNet 2012 moment of CASP,

00:39:11 or protein structure prediction. But back then, that year, what happened was AlexNet. Okay? So when we talk

00:39:17 about the convolutional neural networks, we can talk a little bit about AlexNet, but basically, that's

00:39:24 the beginning of, what should I say, these AI waves. Okay? Before that, I don't think it's a highly

00:39:36 recommended topic for graduate students if they are concerned about their jobs. Okay? But now,

00:39:43 it's probably a different story. Let's see. After that, we see deep learning is, you know,

00:39:52 improving, right? So all kinds of architectures, and some of them will be mentioned when we talk about

00:39:58 convolutional neural networks. And then you see that it's even reaching or beating human levels.

00:40:05 So that's just crazy, right? But this is just analyzing images. Okay? What about others? So if I let you see,

00:40:20 you know, these nice pictures of beautiful human beings, right? We're living in this beautiful world

00:40:28 surrounded by beautiful people, not just beautiful for their faces, but their hearts and their actions.

00:40:35 toward each other. So I wasn't actually very afraid to show these people's pictures

00:40:44 and worry about the intellectual property. Okay? I know there are some databases that give this a

00:40:52 royal free use of images. You can use them without worrying about the intellectual properties.

00:40:58 But here, I don't worry about it at all. Okay? And I will tell you why. In the meantime, if you're on LinkedIn or

00:41:07 Instagram and, you know, one of this profile wants to connect with you, would you think twice?

00:41:15 You would? You would? Some of them. Why? I don't see anything evil from them. Why?

00:41:30 Probably a fake.

00:41:33 What do you mean? So, yeah, so this word, fake, entered our life.

00:41:43 You know, in the daily life, six or seven years ago, there is some, you know, there's a parallel, right?

00:41:51 One is a fake word that kept being used by a president. So in the daily life, you hear about this fake and the fake.

00:42:01 So all of a sudden in our daily world, the use of fake just increased so much, right?

00:42:07 Then there is a tech world. This word fake also got much more usage thanks to a technology, right?

00:42:17 So all these pictures are taken from thispersondoesnotexist.com, okay?

00:42:24 Actually, just as the name suggests, right? So none of them exist. So these images are

00:42:30 created or synthesized, okay?

00:42:32 Actually, there is a new website called thismoleculedoesnotexist.com as well if you're interested in, okay?

00:42:40 So, and if you watch this video and you will see, this is basically, again, let me abuse the word fake.

00:42:52 That's, you know, it looks like Obama talking about something, but it's Obama never made that video

00:42:59 and never talked about whatever is taught. And it was created by the deep fake, okay?

00:43:06 And the technology behind it is GAN, okay? I wonder why, well, this name is really, you know,

00:43:15 is really easy to understand, right? But I wonder if we lived in a different era with a different

00:43:22 president, would we still call them deep fake or something else? But now everybody is using that word

00:43:28 so liberally and, you know, deep fake is affecting our life, okay? So you got to be careful. In the past,

00:43:36 when we, you know, you go to a court and you say that, well, that image is not mine, he photoshopped.

00:43:44 So now you will go there and say, oh, it's deep fake. So you don't know what to trust now, right?

00:43:50 So this is the image creation, okay? But there are way more things than that, right? So, you know,

00:43:57 as we see that other things may be harder to lumped into one domain, but there, you know, we see

00:44:05 unsupervised, self-supervised learning, reinforcement learning, neuroscience-inspired AI, and so on, right?

00:44:14 As I posted today that if I look at what DeepMind has contributed in the past, so many breakthroughs,

00:44:22 but I can see three of their first milestones in each of the following field. The first is AI in games,

00:44:31 that is 2016 AlphaGo, right? The second one is AI in science, that was 2020 AlphaFold,

00:44:44 and the third one is actually today, okay? This is AI in humanity. So if you check the latest Nature

00:44:55 Journal, the feature article is called Ithaca, okay? Ithaca is a computer program that helps

00:45:05 archaeologists recover the inscriptions on the ancient Greek, you know,

00:45:10 stones, right? Those stones, if you are familiar, for instance, Rosetta Stone is a very important

00:45:19 and famous stone, right? If you went to the British Museum, it's out there, right? So it has

00:45:27 three different languages describing the same thing, and that was a key

00:45:31 for translating some older languages, right? So some of there's still a product called Rosetta Stone

00:45:40 for you to learn languages, right? So Ithaca is, you know, is again named after ancient Greek, and

00:45:49 I know so I'm really excited at what AI is creating, but we also have to be aware of what is

00:45:59 that it's negatively impacting us, okay? We will talk about AI and whether AI products are

00:46:12 contributing to or sometimes damaging fairness, right? For instance, right? Or equality, or all of that,

00:46:22 right? So we have seen controversial incidents where AI products just, you know, showed a lot of biases

00:46:30 in their judgment, but maybe those biases are eventually in the human, right? In the data that's

00:46:36 trained, that's used to train the AI products, but eventually came from human, and there are a lot of

00:46:41 pitfalls for us to address, okay? But overall, I think this is just

00:46:46 another beginning of great things, and we are all fortunate to live in this era, okay?
