Lecture 1: Introduction: Bioinformatics and AI

Recording. Download the text transcript.

Automatic transcript; technical terms and equations may contain recognition errors.

00:00:00 It's a blessing to have all of you with such diverse and unique backgrounds.

00:00:05 And I think the field actually needs such diverse talents working together and remove the barrier

00:00:14 across the disciplines because apparently the challenges we are facing are beyond a

00:00:19 single discipline.

00:00:20 So, the first lecture, what are we going to talk about?

00:00:24 There are a few keywords of this course in the title, bioinformatics and algorithms.

00:00:30 And then there are two hidden keywords in this course, data science and AI.

00:00:38 So I'm going to introduce step by step how these are involved and integrated in this course

00:00:45 and give you a better idea to understand what are the topics included in this course.

00:00:53 In the end, we'll give you some additional course information.

00:00:56 The syllabus was already uploaded to Haudi and the Canvas.

00:01:00 And all the lecture notes will be shared before the lecture except this one because I'd like

00:01:06 to keep the season opening a little bit more mysterious.

00:01:10 So I didn't upload these lectures.

00:01:12 But in future, you are expected to see the lecture slides before the lectures.

00:01:16 And the lectures will be recorded, including today's, will be recorded and shared under ENGR

00:01:23 Media Site Collection that's also on Canvas.

00:01:27 We'll collect the assignments and final projects through this Canvas system as well.

00:01:34 Okay.

00:01:35 So, the first thing I would talk about is what are the few keywords?

00:01:41 What is bioinformatics?

00:01:43 I think when I started this course, it's focused on bioinformatics.

00:01:48 But it looks like over the years as it's driven more and more by data science and the concept

00:01:55 is getting broader now than before.

00:01:57 Okay.

00:01:58 But before calling it AI healthcare, let's get more specific.

00:02:02 What is bioinformatics?

00:02:04 Bioinformatics is a conceptual biology, is a conceptualizing biology in terms of molecules, in the sense

00:02:13 of the physical and chemistry, and applying informatics techniques to organize, model, mine, and understand

00:02:24 the information.

00:02:26 The information.

00:02:27 Laser point.

00:02:28 Good.

00:02:29 So the information associated with these molecules on the larger

00:02:34 scale.

00:02:35 So you heard a lot of omics that Nami was saying that's about the scale, right?

00:02:41 In short, bioinformatics is a management information system for molecular biology and has many practical

00:02:48 applications.

00:02:49 So something you want to pay attention is informatics.

00:02:52 It's about how to organize, model, mine, and understand the information.

00:02:57 A lot of the information are buried in the data and needs to be figured out through algorithms.

00:03:04 Okay.

00:03:05 So then if this information are buried in molecules, what molecules are we talking about?

00:03:11 Well, there's a central dogma of molecular biology telling us that there are three important

00:03:17 molecules.

00:03:19 Macromolecules, big molecules in our bodies.

00:03:21 Okay.

00:03:22 They are the most important parts in our body.

00:03:26 This is like the resistor, capacitor, and inductors in the circuits.

00:03:31 You know, you have these three basic parts forming circuits, and then together you can, you know,

00:03:38 build hierarchically the chips, the computers, and then high-performance computing centers,

00:03:45 right?

00:03:46 This is very similar in biology.

00:03:49 These natural systems are also hierarchical.

00:03:51 At the very bottom, you have three most important macromolecules storing and processing the information.

00:04:00 What information?

00:04:01 One of the most important information is genetic information.

00:04:04 That, you know, when you have a newborn, you see that people compliment, oh, you look just,

00:04:09 your baby looks like, just like you or your husband or your wife, right?

00:04:13 So, there's something passing from generation to generation, and that genetic information is

00:04:19 stored in DNA.

00:04:20 So, the DNA, right, the double helix, storing the genetic information gets replicated every

00:04:27 time, from DNA to DNA.

00:04:29 As we speak, your cells are, you know, some are dying, some are splitting, and then the DNA

00:04:35 are being replicated and transferred to the new cells, and you are still you, right?

00:04:40 Okay?

00:04:42 So, this genetic information is kept in a very safe area in our body, right?

00:04:49 So, it's, you know, it's kept within the nucleus of the cell.

00:04:53 It's very safe, okay?

00:04:55 Who can get there and access such information?

00:04:59 There's only one kind of thing that can get there and access the information.

00:05:03 That's called messenger RNA, okay?

00:05:05 So, the messenger RNA gets to the safe room, right?

00:05:09 And then, it actually transfers the information from DNA, right, in the nucleus to ribosome in

00:05:17 the cytoplasm.

00:05:18 So, it's a messenger which transfers the genetic information from DNA to ribosome.

00:05:24 We'll talk more about it.

00:05:26 And the last important molecule is proteins.

00:05:30 So, based on the genetic information stored in DNA and transferred by mRNA,

00:05:36 those are basically used as instructions to make a bunch of proteins that basically do everything

00:05:45 that's happening in our body, right?

00:05:47 You had a hamburger and it's probably already digested after several hours because of some enzymes in your stomach, right?

00:05:56 And you are building muscles and you can expect them, you know, with nutrients and exercise, they grow bigger and bigger, right?

00:06:03 Because of some, you know, again, some proteins in our bodies, right?

00:06:07 We are designing antibodies, right?

00:06:10 And other drugs to fight diseases and these antibodies are proteins.

00:06:16 So, these proteins, mRNAs and DNAs are the three important molecules.

00:06:21 So, these are the molecules that carry the information.

00:06:25 And these are the molecules that we study a lot in this course.

00:06:30 And you will see that these are physical molecules.

00:06:35 But that's in the eye of chemistry or biology.

00:06:40 But in the meantime, these are also multimodal data.

00:06:46 These are data as one-dimensional text.

00:06:50 These are data as two-dimensional graph.

00:06:53 These are data as three-dimensional image or point of cloud simultaneously.

00:06:57 So, that's a blessing to study these molecules through the lenses of physics and data science.

00:07:05 But it's also an opportunity for data science to explore such important molecules from this perspective that you will see later on.

00:07:15 So, now, what systems, right?

00:07:19 So, these molecules talk to each other and, you know, make a system, right?

00:07:25 Actually, seeing the similarity between man-made and natural, nature-made system is not something new.

00:07:34 You know, if you look at this person, you know, this is Norbert Wiener.

00:07:39 Does anyone know about this, you know, absent-minded professor?

00:07:45 If you watch some movies or cartoons, this absent-minded professor, right?

00:07:52 So, I think, you know, this is an urban, you know, a tale or a joke saying that some professor is so absent-minded that one day, you know, they are moving houses, right?

00:08:07 So, the wife knows that he's going to forget about it, right?

00:08:12 So, he tried, so, the professor, after the hours, went back to his old house and finding that the house is already empty, right?

00:08:23 So, then, you know, a little girl came over and say, oh, you know, you have, you know, we have moved to a different place, okay?

00:08:35 And then this is the address, okay?

00:08:37 The professor says, oh, you are such a lovely little girl and thank you for reminding me.

00:08:44 Did my wife send you?

00:08:46 And the daughter said, yes, dad.

00:08:49 So, he's so absent-minded that he didn't even recognize that it's his daughter.

00:08:56 So, there's a lot of jokes about the absent-minded professor and one of the, you can argue who's the stereotype, who's the person?

00:09:07 And Norbert Wiener, he's sometimes rumored to be behind those absent-minded professor jokes.

00:09:16 But anyway, this was 1948, more than 60, 70 years ago.

00:09:23 If you look at the title, Cybernetics, okay?

00:09:27 Cybernetics, if you look at the title, it says, control and communication in the animal and the machine.

00:09:34 So, there are two things I want you to pay attention.

00:09:36 One is that the animal and the machine are talked about together, okay?

00:09:42 Secondly, it's about control and communication.

00:09:45 If you think about bioinformatics, it's about information flow.

00:09:48 You have to understand how the information is communicating and how that can be controlled.

00:09:54 So, that is already a system view back in 1948, okay?

00:10:00 As a side note, how artificial intelligence as a word started actually, right?

00:10:08 It was actually because part of the reason was that cybernetics was AI before AI as a word was created, okay?

00:10:20 But at the same time, those people in the Dartmouth conference also want to make sure that they distinguish what they do from the past,

00:10:29 including the cybernetics.

00:10:31 So, that's part of the reason artificial intelligence as a word was newly created,

00:10:37 to differentiate what they do from their own and cybernetics.

00:10:42 We'll talk about artificial intelligence briefly today, but in more details around lecture 14, okay?

00:10:48 So, sorry, can I go back?

00:10:54 Yes.

00:10:55 So, in other words, we see that, you know, you can already have a system view, as I said earlier,

00:11:00 that hierarchically you can start with a genome that makes molecules and molecules talk to each other and form cells,

00:11:08 and multiple cells form organs, and organs communicate to form human bodies,

00:11:13 and then people interact to form social networks.

00:11:17 So, in that sense, the animal and the machine are both hierarchical, right?

00:11:22 So, that's part of the reason that we are all in this room, because it's about the information flow.

00:11:29 It's about the communication and the control, and it's more and more driven by data science.

00:11:35 So, that's part of the reason that unite all of us, even though we have very diverse background, okay?

00:11:42 So, actually going from molecular biology to systems biology is one of the biggest, you know, difference about biology in the 21st century.

00:11:52 You know, in the 20th century biology, you can spend your whole life studying a single molecule and ask what it does.

00:12:00 But now people are more and more looking at these molecules in an integrative approach.

00:12:07 In other words, you know, they are looking at all these molecules in the system.

00:12:11 That's why you have all these omics study, right?

00:12:14 Proteomics, genomics, transcriptomics, and so on, right?

00:12:19 Metabolomics, and so on, right?

00:12:21 So, we are more and more looking at biology from the systems perspective.

00:12:28 And the way we are looking at the biology from the systems perspective is more and more data driven, okay?

00:12:36 And computational biology is becoming more and more driving, taking a bigger and bigger role driving biology.

00:12:46 At the beginning, it's more validation, you know.

00:12:48 Biology experiments generate data and computational biology or bioinformatics analyze data.

00:12:54 But nowadays, things are getting different.

00:12:57 You see more and more bioinformatics are designing experiments, actually, to collect data.

00:13:03 They are trying to address the question, to gain insights or knowledge, what data should I collect?

00:13:13 So, the bioinformatics is now more and more, you know, taking the front seat rather than the back seat.

00:13:19 So, that's the difference we have to realize.

00:13:22 You know, we are not only going from the molecule level to the systems on a level.

00:13:25 We are not only going more and more quantitative and data driven, but also we are more and more better interacting with the wet lab experiments.

00:13:34 So, those are the major challenges.

00:13:36 Sorry, those are the major changes that have happened in the recent years, okay?

00:13:42 So, then what data, right?

00:13:45 There are too many types of different data, right?

00:13:48 We can talk about all kinds of omics.

00:13:50 But I can just give you a few, you know, examples, right?

00:13:54 And we'll study some of them very heavily in this course.

00:13:57 And in the final projects, you can explore some other data even more heavily, right?

00:14:02 So, there's this genetic information stored in DNA.

00:14:05 That's the DNA sequence, right?

00:14:07 So, we have how many base pairs in our genome?

00:14:10 Anyone roughly knows?

00:14:14 Just a trivial question.

00:14:17 How many base pairs in human genome?

00:14:20 Thousands, tens of thousands?

00:14:23 Hundreds of thousands?

00:14:24 Hundreds of thousands?

00:14:25 How many?

00:14:26 Hundreds of thousands.

00:14:27 Hundreds of thousands.

00:14:28 Yes?

00:14:29 Maybe 30,000.

00:14:30 Okay, something like that.

00:14:35 And then at each position, you have four choices of ATGC.

00:14:40 So, you have, commentatorially, you have four to the power of 30,000 or what you said, hundreds of thousands.

00:14:47 Right?

00:14:48 So, we have a huge astronomical number, right?

00:14:52 So, of course, not every sequence is possible, but that stores a lot of diverse information about what makes what we look like, right?

00:15:01 And also, what disease might be carried and can be overcome.

00:15:06 And then, the DNA information was carried, right?

00:15:09 Genetic information was carried by mRNA.

00:15:12 And then, this information is used as a Bible to make proteins, right?

00:15:17 So, these proteins have one-dimensional sequence, but they also adopt three-dimensional structure to function, right?

00:15:24 So, these are the molecules.

00:15:25 And then, you can look at even more data, high-throughput experiment data.

00:15:29 For instance, you can look at the gene mutation, methylation, copy number variation, gene expression level.

00:15:36 So, you know, what's the quantity of every gene expression?

00:15:40 You can look at the experiments to tell what proteins are talking to what proteins, right?

00:15:45 And then, you can use RNA silencing, right?

00:15:48 To find out, you know, to silence some genes and see how the systems respond.

00:15:55 Or you can administer some drug, right?

00:15:59 To see how this system responds.

00:16:02 In other words, you can use, you know, gene editing like RNAi or CRISPR.

00:16:08 Or you can use small molecules like drugs as perturbation to your system, right?

00:16:13 Which can be in the mouse.

00:16:14 Then, you can observe how the phenotype changes, right?

00:16:17 And then, there are a lot of imaging data, actually, nowadays, as well.

00:16:22 As you will see later on, imaging data is actually the inspiration for the first wave of AI healthcare.

00:16:32 Because imaging had a revolution in data science.

00:16:35 And then, naturally, after its success, you know, in object identification, it quickly went into the healthcare.

00:16:45 So, why now more than ever?

00:16:48 Because we are generating way more data than before.

00:16:51 It's that simple.

00:16:52 If you look at, you know, early 2000s when the human genome projects started, it cost hundreds of millions of dollars to get a genome.

00:17:04 Nowadays, I think it stopped at 2015.

00:17:07 But I think it's below $1,000, okay, nowadays, to get the human genome for a single person.

00:17:16 So, this is just one kind of data we are getting much faster and cheaper right now.

00:17:23 This is just one type of data.

00:17:24 And then, imaging and all other kind of technological revolution made sure that we have more and more data, right?

00:17:31 If you look at, you know, computer vision, how it got, you know, revolutionized.

00:17:36 One of the biggest reason is ImageNet and the data, right?

00:17:39 It's a data-driven research you cannot, you know, just don't have or, you know, you cannot suffer from, you know, low quality or small quantity data, right?

00:17:51 So, that's one issue.

00:17:52 Of course, compared to other kind of domains, bioinformatics data are still more or less much more challenged, especially in the case of the labeled data.

00:18:03 We'll talk more about that, right?

00:18:05 And then, this is just data, right?

00:18:08 In the meantime, you know, the computational resources are getting more and more.

00:18:15 GPU, right?

00:18:16 Now more than ever.

00:18:17 And the way they integrate with neural networks, right?

00:18:22 And then computing powers, right?

00:18:24 Our, you know, high performance computing are getting more and more powerful.

00:18:27 So, that's another reason, right?

00:18:29 If you look at the proteins similarly, right?

00:18:32 You see the number of protein sequences, right?

00:18:36 Growing from 1980s to 2010s, right?

00:18:41 And then on the right, you have the number of protein structures that is growing, right?

00:18:45 So, actually, in year 2021, the structure biology has seen a major revolution due to artificial intelligence.

00:18:56 That's another thing you will see later on.

00:18:59 So, and then, you know, other data example, for instance, this omics, right?

00:19:05 You have all these, you know, you see this DNA seek, fair seek, attack seek, that the hyphen seek means the high-throughput sequencing, okay?

00:19:16 And what's ahead of it is other kind of assay.

00:19:19 So, usually, by combining high-throughput sequencing and some other experiments, you end up having a lot of data.

00:19:25 Describing different aspects of biology.

00:19:31 It's like those multiple blind people feeling an elephant, okay?

00:19:36 So, for instance, one piece could be protein-protein interactions.

00:19:40 Another piece could be gene expression, right?

00:19:43 And one of the omics study is trying to integrate across omics and to have a better idea about what this elephant look like.

00:19:53 Not just the dead elephant.

00:19:55 It's a moving and living elephant, okay?

00:19:57 It's not a still elephant being felt by multiple blind people, okay?

00:20:02 So, that's a challenge.

00:20:04 So, that's from ENCODE, for example.

00:20:06 So, why bioinformatics?

00:20:09 And if you want to use a better cool name, data science, AI, health, whatever you call it, I think you have to ask why we need this kind of work, right?

00:20:21 So, I think the central idea is that we're trying to discover new biological insights, right?

00:20:27 We are trying to understand why the biology works the way it works.

00:20:32 And when it does not work, right?

00:20:35 For instance, when we have disease or when we need more energy or we need these crops to be more resilient to insects or fungus, then how can we re-engineer these biological systems, right?

00:20:51 And then in this process, you have to talk about a lot of things about data storage, retrieval, and analysis because it's more and more driven by data science, right?

00:21:04 So, you can look at the comic, the cartoon after the lecture and see what I'm trying to say here.

00:21:10 Okay, so, and why bioinformatics more, you know, we have, but, you know, that's about what biology needs, right?

00:21:21 Then you may think why AI needs bioinformatics or healthcare.

00:21:26 I think it's a push and pull, okay?

00:21:29 It's a two-way communication between bio and AI.

00:21:34 I just said that what AI could do to bio, right?

00:21:37 But what bio could do for AI, you have to understand that bioinformatics joining social and finance are providing one of the biggest data corpus to AI, right?

00:21:53 And in the meantime, we do have both big and small data, okay?

00:21:58 It provides more applications.

00:22:00 It provides bigger societal impacts.

00:22:02 More and more AI, IT industries are investing in healthcare, right?

00:22:09 And in the meantime, you have to understand that these applications, especially in bioinformatics, are constantly providing new challenges and inspirations for a paradigm shift needed for foundational AI.

00:22:28 In other words, it may be hard for you to think about a foundational challenge to improve the graph neural networks, but these applications can tell you that, oh, you have to address this heterogeneous graph, you have to address these hierarchical graphs as they arise from bioinformatics.

00:22:48 So they give you a lot of challenges, but also inspirations.

00:22:52 So when you see a river, if you see a river, then you should be happy if you are a bridge builder, because that gives you a place to build a bridge, right?

00:23:03 It depends on who you are.

00:23:04 So if you see challenges, you should be happy because they are providing inspirations.

00:23:09 And I can talk about the sub-areas of bioinformatics, but I think this is a little antique, because later on you will see that we have to address these sub-areas in a joint way.

00:23:23 But basically you have the sequence analysis, you have the structure analysis, you have the database, that's a traditional.

00:23:29 I think the new waves are actually smearing the boundaries, so let's not think about this too much.

00:23:35 But at the very beginning when the course title was made, it was about structure bioinformatics, but I think it's much broader than that now.

00:23:47 And then we talk about one-dimensional sequence and why we care about the three-dimensional structures, because they tell us what each part does and why it does so.

00:23:57 So we'll talk more about those as well.

00:24:01 Then why us engineers or data scientists are interested in bioinformatics or structural bioinformatics.

00:24:09 One thing is it's highly interdisciplinary, right?

00:24:13 It needs people more than trained in one area.

00:24:19 And we engineers or data scientists are good with two things, systems and data, right?

00:24:25 We know how to address, you know, or design systems to process signals, right?

00:24:32 We know how to address the challenges associated with such data, right?

00:24:37 As a data scientist, you want to deploy the data-driven models in specific applications.

00:24:43 That's why when computer vision first made early success, they quickly went through the medical imaging field, right?

00:24:50 So because they need bigger societal impacts and they need to deploy their products in certain sceneries, right?

00:25:00 As an algorithm developers, you get inspirations from these applications.

00:25:05 So these are the reasons that why you might be interested in it, right?

00:25:08 But there are more reasons as we will see later on.

00:25:11 These are just some early examples why engineers are actually contributing to the birth of synthetic biology.

00:25:18 Synthetic biology is like forward engineering.

00:25:22 That is like you are designing parts that work together in the system, right?

00:25:29 In early 2000, James Collins, you know, basically constructed a genetic toggle switch in a living system E. coli, right?

00:25:39 This is, you know, a bistable system that you study in control, but then it gets realized in biology, right?

00:25:47 So, and then you can engineer particles to deliver antibiotics and you can design better bacteria to have to brew better taste beer, right?

00:26:03 You can do a lot of different things, right?

00:26:05 So, and another key word is algorithms, right?

00:26:11 Bioinformatics, why we care about it.

00:26:13 What algorithms do?

00:26:15 I think I can roughly split them into three purposes.

00:26:20 And the first one is analysis.

00:26:23 You know, you have a bunch of data.

00:26:25 Your questions are what patterns are there and why are there such patterns?

00:26:30 So this is analysis, okay?

00:26:33 The second is prediction that is nowadays done a lot by neural networks.

00:26:38 The question is what would happen given this?

00:26:42 So if I give this input, what would happen?

00:26:44 If I make a mutation on this gene in this way, what would happen to this, you know, likelihood of having the breast cancer, for instance?

00:26:53 And the last one is the inverse direction compared to prediction.

00:27:00 That is, what should I do to make something happen?

00:27:04 That is, to make some outcome occur, what input should be provided?

00:27:10 Okay?

00:27:11 So given y, what should be desired y, what x should be provided?

00:27:17 Nowadays, this is done more and more by generative models in deep learning, okay?

00:27:23 So what algorithms are involved, we are talking about optimization, but I think optimization and machine learning are two sides of a coin, to be honest.

00:27:36 Because most of the machine learning or deep learning problems can be formulated as optimization problems, and they have to be solved by numerical optimization, okay?

00:27:45 And of course, as I said, that when we do data analysis, when we do prediction and design, we are increasingly using deep learning.

00:27:56 For instance, you use representation learning to figure out the pattern.

00:27:59 You use, you know, discriminating models to predict the outcome, the output given the input.

00:28:06 And you use the generated models to predict the input given the desired output, right?

00:28:12 So it's becoming increasingly enabled by deep learning.

00:28:17 And the concept of deep learning is actually getting broader and broader now.

00:28:23 Now, let's talk about the new chapters, right?

00:28:26 So machine learning, deep learning, and AI.

00:28:30 What's the relationship between them, right?

00:28:33 I think nowadays, the hottest word among the three is probably AI, right?

00:28:42 Right?

00:28:43 But most of the time when talking about, people are talking about AI nowadays, they are talking about deep learning.

00:28:51 But AI is much bigger than deep learning.

00:28:54 We'll introduce in more details about the history and sub areas of AI.

00:29:03 But it's, as you know, as I said, that even in the era of cybernetics, you can argue that it's already AI.

00:29:10 But as the official birth was in 1980s after the Dartmouth Conference, right?

00:29:19 And it includes multiple areas and the one era of machine learning got popular and flourishing from the 1980s.

00:29:29 And one of the earlier successful machine learning models are, believe it or not,

00:29:34 neural networks already exist, way 1980s, way back, right?

00:29:40 And back then, it was support vector machines beating neural networks, right?

00:29:46 Why?

00:29:47 Because neural networks cannot be solved as optimization problem cannot be solved exactly.

00:29:53 But support vector machines can.

00:29:55 And the support vector machines can deal with the dimensionality issue much better than neural networks, right?

00:30:01 And then neural networks, or deep neural networks, got its popularity actually from the 2010s, right?

00:30:11 After the success in one ImageNet competition, right?

00:30:18 Because people were used to the ImageNet competitions year after year sort of, you know, saturation performances.

00:30:26 But all of a sudden, one year, the performance was improved much more, right?

00:30:31 So that's the moment that people start to pay attention to deep learning and everybody is using deep learning, right?

00:30:37 So this is the relationship between AI, machine learning, and deep learning.

00:30:42 So it's apparently rippling from the spam detection or object detection all the way to healthcare as well, okay?

00:30:52 So the artificial intelligence for biological intelligence, I mean understanding and engineering biomolecules and the biosystems, right?

00:31:02 So a few years ago, you already see that the economist reporting how to determine a protein's shape, right?

00:31:12 So a few years ago, you see, this was 2018, right?

00:31:18 Deep learning for biology as a technology feature published in Nature, right?

00:31:27 In the past year, this is actually the year of AI revolution to structure biology.

00:31:34 Some people even said that maybe structure biology is reaching an end.

00:31:39 I doubt it, but it really says something about the revolution.

00:31:45 So the method of the year 2021 is protein structure prediction.

00:31:51 That is, you have the protein one-dimensional sequence, and you try to predict its three-dimensional structure

00:31:57 in order to understand its function and in order to redesign it for different properties and so on.

00:32:03 So again, just like ImageNet, there is a competition called the CUSP, critical assessment of structure prediction.

00:32:11 Again, people are used to the small incremental improvements year after year.

00:32:16 But, you know, 2020, there is a performance from a company, DeepMind, right?

00:32:26 Their model is called AlphaFold.

00:32:28 If you follow AlphaGo, you know that this is the same company, right?

00:32:32 That invented AlphaGo, who beat a human Go champion way earlier than what we had anticipated.

00:32:40 And it's the same company who designed AlphaFold version 2, which improved the performance much more, right?

00:32:48 So, you may read the first issue of this year for Nature Methods.

00:32:54 It actually features multiple comments from the DeepMind group, from the David Baker group, from the other groups.

00:33:02 Now, people are using the predicted protein structures more and more often as real data and study their impacts on biology.

00:33:13 So, this is an exciting era to live in if you are into data science and healthcare.

00:33:21 So, but in general, if you talk about AI in healthcare, there are many different things that you can do, right?

00:33:28 You can do the robotic surgery, right?

00:33:31 So, you can control how the robotics move during the surgery.

00:33:35 You can enable the remote surgery, especially for the non-urban area, you know, where the infrastructures are not available, right?

00:33:47 You could have virtual nursing assistants.

00:33:51 Nowadays, you probably, when you are making medical appointments or when you are, you know, communicating with your doctors,

00:33:58 very often what your input is processed by a virtual nursing assistants first, right?

00:34:04 What is behind that is because of what technological revolution in AI?

00:34:10 Natural language processing, right?

00:34:18 And then there's medical image analysis, again, this naturally empowered by computer vision.

00:34:24 And then the latest wave of AI to healthcare is now, it's just crazy, in the past three years, more and more companies doing AI healthcare are venturing into drug discoveries directly, okay?

00:34:37 Which is a process that is costing billions of dollars and taking 10 years and have a very low success rate.

00:34:46 And then they are looking into AI.

00:34:49 And there are other kinds of things involved as well.

00:34:52 So, drug discovery is one thing that we try to talk a lot in this course, right?

00:34:58 So, the prospect is that you see more and more, right?

00:35:02 So, this was analysis, right?

00:35:05 The market of AI medicine was valued at 719 million in 2017.

00:35:11 It's estimated to reach 18,000 million at a cumulative annual growth rate of 49.6% from 2018 to 2025.

00:35:21 I think COVID affected that trend in two ways.

00:35:26 In one way, it did make things slower, but in the other direction, it also made things faster.

00:35:33 Think about the messenger RNA based vaccines.

00:35:36 How long would it become a reality if there were no COVID, right?

00:35:40 So, it's doing things in both ways.

00:35:43 A lot of startups are, you know, are surging in artificial intelligence for drug discovery now, okay?

00:35:51 As I said, the first wave is computer vision going to healthcare.

00:35:56 So, a lot of the medical imaging analysis was replaced by AI, right?

00:36:01 I think Jeff Hinton even said during the interview that you shouldn't consider taking education in radiologists,

00:36:11 being a radiologist anymore because that profession is going to disappear.

00:36:15 It's going to be replaced by AI, right?

00:36:17 I think that's over estimation.

00:36:21 It hasn't happened yet, right?

00:36:22 It says that a lot of things are not fully solvable by AI.

00:36:27 Maybe not even formulated for AI yet, right?

00:36:30 The second wave is natural language processing, right?

00:36:33 So, it's, you know, I'm not saying that it's much later than vision, but if you remember IBM, right?

00:36:39 Watson winning what?

00:36:42 The jeopardy, right?

00:36:44 Against the humans, right?

00:36:46 That's, you know, early natural language processing success, right?

00:36:50 They actually started collaborating with MD Anderson right away, right?

00:36:54 To analyzing the medical records and, you know, giving more diagnosis and so on, right?

00:36:59 So, there's success, but there are also failures, right?

00:37:03 But you cannot argue that, right?

00:37:05 Nowadays, electronic health records are being processed much more often by AI, right?

00:37:13 Natural language processing.

00:37:15 And, you know, this is one example.

00:37:19 How many of you know Daphne Kohler?

00:37:22 How many of you have taken a course in probabilistic graphical models or read her textbook,

00:37:28 probabilistic graphical models?

00:37:30 This is a computer scientist, Daphne Kohler.

00:37:33 So, four years ago, three or four years ago, they started a company, right?

00:37:39 A startup.

00:37:41 And Daphne Kohler's, in her eyes, she's saying that, what is machine learning for in-sitro's purposes?

00:37:50 In-sitro is a company that they started.

00:37:53 Her definition is, it's a task of building machine algorithms that, given access to a very large data set, are able to provide prediction tasks with very high performance.

00:38:05 So, in this case, what their company is doing is predicting what will happen when a chemical, a potential medicine, is added to the human body.

00:38:17 What will it do?

00:38:18 Okay?

00:38:19 So, they got actually investment from Gilead, a major pharmaceutical company, and in two phases, right?

00:38:26 In the first phase, they got $50 million to find up to five proteins that can be targeted for some disease.

00:38:36 And then the second stage of 35 million is to find small molecules that can, you know, interact with those protein targets for the desired therapeutic purposes, right?

00:38:48 But these are very old numbers.

00:38:50 I think they already got around C or more investments, okay?

00:38:55 So, they are doing very well.

00:38:57 This is just one example.

00:38:59 And there are too many of them nowadays, okay?

00:39:03 But you got to be careful, okay?

00:39:07 If you say that, oh, this is a 10-year drug discovery process, and now there's AI, it's going to be much faster.

00:39:13 Well, you got to be careful.

00:39:15 You got to be careful.

00:39:16 A lot of bubbles are burst because they overestimated their tools power but underestimated the challenge from the field, right?

00:39:26 If you look at what AI is doing in drug discovery, it's still mostly focused on the low-hanging fruit.

00:39:33 It's very good at working on the chemistry data because there are a lot of data, a lot of the labeled data in chemistry, right?

00:39:41 But for the latter stage of drug discovery, there are a lot of biology.

00:39:46 There's a lack of labeled data in biology, and sometimes, you know, they are not even conceptually understood.

00:39:52 So, in those cases, AI is slowly making impacts.

00:39:57 So, those are the roadblocks.

00:40:00 It's still not universally accepted by the healthcare professionals.

00:40:07 Just one simple thing.

00:40:08 If you want to convince the healthcare professionals to use not the, you know, paper questionnaire but instead of use, you know, electronic devices to collect patients' feedback, that already takes a lot of efforts to convince them.

00:40:29 It's natural that, you know, they sometimes hesitant to accept, especially after a lot of failures, including the collaboration between IBM Watson and MD Anderson, right?

00:40:42 And then also, it's now focusing more on the low-hanging fruit than the real challenges, which makes sense because you don't want to, you know, work on, you have to pave the way to solve the internal challenge.

00:40:58 You cannot just, you know, solve the ultimate challenge right away.

00:41:03 I think that is good.

00:41:04 Nowadays, a lot of benchmark data sets, a lot of data challenges, Kaggle challenges or critical assessments are being developed so that these AI in healthcare can really develop.

00:41:18 Okay, otherwise, it cannot even be evolving.

00:41:22 So, let's recall, you know, why by informatics we said that, you know, we do need these new challenges and new inspirations, right?

00:41:33 for the paradigm shift needed for the foundational AI.

00:41:37 So, in other words, your AI is not only, you know, use targeted, right?

00:41:43 You're not just applying your AI to some field.

00:41:47 You also have these AI that are use initiated.

00:41:51 So, that's one thing you have to keep in mind, okay?

00:41:54 So, it goes both ways.

00:41:55 It's a push and pull process between AI and healthcare.

00:42:00 So, in the end, I'd like to say that, you know, we spend a lot of time.

00:42:07 So, during the conversion, there's some error message.

00:42:10 I apologize for that.

00:42:12 I want to say again that what we are doing here in this course a lot is using one example and one focus study to illustrate the innovation and the history under the innovation.

00:42:29 the innovation in two directions.

00:42:31 First of all, we are dealing with molecules that are of multi-modality data.

00:42:37 As I said that this protein is a one-dimensional sequence like atom, right?

00:42:43 So, and then if you think about what is A, it's alanine that consists of multiple, sorry, multiple atoms, right?

00:42:57 And you also have, so that's a one-dimensional sequence and three-dimensional structure.

00:43:03 And then depending on what amino acid or letters are close to each other, you could represent it as a graph or adjacency matrix, right?

00:43:15 So, naturally, our molecules, especially proteins, are available in multiple modalities.

00:43:24 So, it's a text, it's a graph, it's a 3D point cloud, and they are increasingly treated that way in AI.

00:43:33 The reason that AlphaFold 2 made a revolution in year 2020 and 2021 is exactly respecting the fact that protein structure prediction problem is a multi-modal machine learning problem.

00:43:50 So, in this sense, one of the key AI aspect that you can get from this course is to learn multi-modal learning and generation.

00:44:02 Learning is more about discriminating models and generation is more about the generative models.

00:44:10 Another fact is that these data are actually governed by physics or principles.

00:44:17 So, even though neural networks are universal function approximator, it's hard sometimes to teach it everything through data.

00:44:31 Think about convolutional neural networks, if you just try to teach that it should be rotationally invariant,

00:44:39 if you do that by using a lot of data aggregation, it's very low efficiency, right?

00:44:46 And without much success considered.

00:44:48 So, in other words, you have to learn how to integrate principle and data.

00:44:54 So, in other words, you have to make sure that this is enabled by smart data and it can be enabled by the smart way of integrating principle to the data-driven model.

00:45:10 The fact that the model is data-driven doesn't mean that everything can be learned effectively purely through data.

00:45:19 A lot of things need to be engineered very smartly.

00:45:22 For instance, you may consider changing the architecture so that it doesn't have to learn from data the rotational invariance.

00:45:31 You can directly put the rotational invariance into the neural networks so that the underlying function family is already satisfying that property.

00:45:44 Instead of forcing a much broader family of function to learn that from a huge amount of data, right?

00:45:51 So, that's another thing I want you to learn from this course.

00:45:55 One is multimodal learning and generation.

00:45:57 The other is integrating principle and data.

00:46:00 Let me be more specific.

00:46:02 So, we are using protein structure prediction as a focus example to learn this evolving from principle-driven to the data-driven approaches,

00:46:14 approaches to learn this multimodal learning and generation.

00:46:19 So, and in this process we do three stages.

00:46:22 At the beginning, we'll pay more attention to the traditional methods that are more principle-driven.

00:46:29 Although, there were a lot of data-driven methods there already.

00:46:33 So, there we'll pay a little bit more attention to optimization, including those linear dynamic programming and numerical optimization methods.

00:46:43 So, those probably will cost us seven to nine lectures.

00:46:48 And then, using the opportunity to start with protein sequence, the structure and data, and we'll try to predict the protein function.

00:46:56 Using that case study, we're trying to explain a little bit more about machine learning and more importantly,

00:47:03 how do you integrate data in machine learning, not the deep learning yet, and how do you diagnose your machine learning models.

00:47:11 So, more than half of the lectures will be focused on the data perspectives for protein structure prediction.

00:47:19 Okay?

00:47:20 Again, this is starting from the sequence, structure, and data for the function.

00:47:24 So, we'll try to view the proteins as an image and see how the image data is addressed in computer vision through convolutional neural networks,

00:47:36 and what makes our image special.

00:47:40 We'll try to address the protein sequence as text, and then learn how the natural language processing is accomplished through RNNs and transformers.

00:47:52 We'll also learn proteins as three-dimensional data, and then how the three-dimensional data can be learned through rototranslation of equivariant neural networks.

00:48:03 In the end, we'll also have the graph neural networks interspersed between, but in the end, you have the 1D, 2D, and the 3D data jointly learned.

00:48:15 That's a multimodal learning we're trying to emphasize.

00:48:20 And one, when we talk about the sequence data or the natural language processing, we'll talk a little bit more about the so-called protein language models.

00:48:32 That you start with the protein sequence, and you try to predict a lot of things about the protein.

00:48:36 [Uncertain recognition] And I'm just going to say, you're just going to see the next one, you can think of the next one, you can think of the better.