Lecture 11.1: Protein Function Prediction
Recording. Download the text transcript.
Automatic transcript; technical terms and equations may contain recognition errors.
00:00:00 So this will be the end of the optimization module.
00:00:03 Any questions so far?
00:00:06 Okay, now I'm gonna start the next module
00:00:13 using a different problem to motivate this formulation.
00:00:19 Okay, so we're talking about machine learning,
00:00:21 but you know optimization and machine learning
00:00:23 are two sides of a coin.
00:00:25 So I'm gonna start with the protein function prediction.
00:00:29 So far we're doing sequence and structure,
00:00:31 but don't forget function, okay?
00:00:35 So where we are, we are right here, okay?
00:00:40 We're gonna use this to explain the basic concepts,
00:00:44 but I think many of you already know machine learning already,
00:00:48 so I'm gonna go through this part really quickly,
00:00:51 and sometimes I will just directly ask you,
00:00:55 you know, what do you recollect for those concepts, okay?
00:00:59 And at the end of today's lecture,
00:01:01 I will ask you a question about support vector machine,
00:01:04 so please don't let me forget about it.
00:01:07 Please remind me, okay?
00:01:10 Okay, so, motivation, right?
00:01:13 So we're now moving to the next question, right?
00:01:17 Protein function prediction.
00:01:18 So why, what, and how?
00:01:21 Okay, and then how this motivates,
00:01:24 how the challenge motivated machine learning models, okay?
00:01:28 So why, we know that there is a gap between
00:01:32 the sequentially determined and the structurally determined
00:01:35 proteins, right?
00:01:36 But also the gap between the sequenced determined
00:01:40 and the structurally annotated, sorry,
00:01:42 functionally annotated proteins are also wide, okay?
00:01:45 For instance, this is the green one, Swiss prot, okay?
00:01:50 This is the manually annotated protein sequences, okay?
00:01:54 Those sequences are already manually annotated
00:01:57 with the function, okay?
00:01:59 But there are way more sequences, right?
00:02:01 In tremble, which include a lot of sequences
00:02:03 that haven't been annotated or just annotated automatically, okay?
00:02:06 So these three data cells are getting bigger and bigger.
00:02:13 Protein sequences that have been structurally determined,
00:02:17 protein sequences that have been functionally annotated,
00:02:21 and protein sequences that have been collected, okay?
00:02:26 So, and if you look at this,
00:02:30 this is the same thing.
00:02:33 This genome-wide protein function prediction,
00:02:38 this is old data, but still,
00:02:40 more than 40% of the known sequences, right?
00:02:45 In our species, even after you exhaust
00:02:48 the homology-based method for annotation,
00:02:51 you still end up having like 40%, okay?
00:02:55 40% of the molecular functions unknown.
00:02:59 So now, the question is,
00:03:02 how do we do that?
00:03:04 Again, why is this a computational problem?
00:03:06 You told me that sequence is 1D,
00:03:08 structure is 3D, I understand.
00:03:11 But how do you even describe a function?
00:03:15 I don't even know what's the dimension, right?
00:03:17 So like what we did before,
00:03:19 we looked at the sequence structure relationship before,
00:03:23 and ended up knowing that there are some,
00:03:27 you know, heuristic rules,
00:03:28 then we can look at the 30% as a cutoff, right?
00:03:32 And figure out whether we are in the safe zone or not,
00:03:34 right?
00:03:36 Here, I'm gonna do something similar,
00:03:38 but we're gonna, you know, talk about more about it.
00:03:41 So sequence, so in general, there's no golden rule, okay?
00:03:45 But there are some, you know, heuristics,
00:03:47 like we said before, safe zone is 30%, okay?
00:03:51 And we knew that the structure is more conserved than sequence,
00:03:54 right?
00:03:55 Like, this can be explained by this bullet points
00:03:58 that we have learned before, okay?
00:04:02 And it's like the one versus 100, okay?
00:04:07 Then what about the sequence versus function?
00:04:09 Well, the safe zone is now at about 40%.
00:04:13 If the identity is about 40%,
00:04:16 then the sequences tended to have similar structures
00:04:20 and the functions, okay?
00:04:22 Again, it's a heuristic.
00:04:25 What about the structure versus function?
00:04:28 Well, it's more complicated and less well known,
00:04:31 but because one issue is that I cannot say that similar structures
00:04:35 give you similar functions,
00:04:37 because I haven't described the functions yet.
00:04:39 One structure can have multiple functions, right?
00:04:44 Right?
00:04:45 So this is another issue,
00:04:49 and also your data might be biased because, look,
00:04:53 all the protein sequences that have been structured determined
00:04:55 are already biased.
00:04:57 Those structures are structures that the current technology,
00:05:01 like NMR, X-ray can collect, right?
00:05:05 So that data set is already biased,
00:05:08 so I cannot really derive my conclusion from such small data set,
00:05:12 right?
00:05:14 So let me give you one example, okay?
00:05:17 So this is one structure, multiple functions, right?
00:05:21 So this protein is an enzyme.
00:05:24 It's called the glucose 6-phosphate isomerase.
00:05:28 When you see an ASE at the end,
00:05:30 you know that it's kind of an enzyme
00:05:32 that makes some chemical reaction much faster.
00:05:36 And this enzyme is basically converting the glucose 6-phosphate substrate
00:05:43 to fructose 6-phosphate, which is a product.
00:05:48 This is what this enzyme does, but it's just the one function, okay?
00:05:53 So if it's forming a dimer, this is a dimer, okay?
00:05:57 You have this one monomer in the front, one monomer in the back, right?
00:06:02 It's symmetric.
00:06:04 So this dimer in the cell is accelerating this chemical reaction, okay?
00:06:13 But it could be a monomer outside of the cell, right?
00:06:18 Which act as neuroleukins, cytokines, and so on.
00:06:22 So they could have different functions, okay?
00:06:25 So now you see there's a problem, right?
00:06:28 Functions, first of all, there's no natural dimensionalities to begin with.
00:06:33 Secondly, function is not just sequence and the structure determined.
00:06:38 It's also context dependent, right?
00:06:42 If it's in the cell or it's outside of the cell,
00:06:45 if it's a dimer versus a monomer, right?
00:06:47 Or, you know, the function could be what partners or protein interactions you form, right?
00:06:55 So that's a complication.
00:06:57 So then how you define functions?
00:07:00 The way you define protein functions,
00:07:03 how many of you have heard of this gene ontology?
00:07:07 Good.
00:07:10 So it's okay, I'm just trying to figure out.
00:07:12 So gene ontology is basically consisting of three ontologies.
00:07:19 Molecular function, biological process, and cellular component, okay?
00:07:25 Molecular function says, what does this protein do?
00:07:30 Biological processes ask, what processes is it involved in?
00:07:36 And the cellular component asks, where does that happen?
00:07:39 Okay, so this is why we have three ontologies, okay?
00:07:44 The reason that we call it the gene ontologies is because proteins are gene products, right?
00:07:51 Right, remember, it's translated.
00:07:54 So if you need to know what ontology means, we have a lot of ontologies.
00:07:58 It's a formal representation of a body of knowledge within a given domain, okay?
00:08:04 So there are a lot of, it's describing a lot of relationships between classes or terms, okay?
00:08:11 Okay, here we have three ontologies, right?
00:08:15 Okay, as I said, cellular component, biological process, and the molecular functions, okay?
00:08:20 This is one of the earlier versions, but you can go to geneontology.org,
00:08:24 and you can find out that in each ontology we have certain number of nodes,
00:08:31 and each node is one specific molecular function, for instance,
00:08:38 and each node may correspond to multiple proteins.
00:08:43 In other words, multiple proteins may all reside or share that function node, okay?
00:08:50 So, for instance, right, this is a simple illustration.
00:08:58 We focus on the biological process rather than these other two, okay?
00:09:05 Let's start with the biological process, then we basically see that there are a lot of nodes
00:09:10 and a lot of arcs or edges, right?
00:09:13 So there are actually many different kind of relationships,
00:09:17 relationships, but in general, as you start from, now this is like an upside down tree, okay?
00:09:25 If we only consider certain relationships.
00:09:28 So on the top, you have the root.
00:09:31 As you go down to the leaf, you see that if you follow a path,
00:09:39 the function is getting more and more specific, isn't it?
00:09:44 Isn't it?
00:09:45 So, for instance, here is a regulation of biological process.
00:09:50 This is a positive regulation, right?
00:09:53 And for in this case, pigmentation, and this is pigmentation during development, and so on, right?
00:09:59 So it's getting more and more specific.
00:10:01 But it's not general because it depends on what kind of edges you consider, okay?
00:10:06 So it's like an instance, right?
00:10:11 So in this case, pigmentation is a biological process.
00:10:15 Pigmentation during development is a pigmentation.
00:10:18 So I means is a, but there are other kind of relationships, okay?
00:10:23 If you only consider is a, this relationship, then this graph can be roughly regarded as a DAG,
00:10:33 a directed acyclic graph.
00:10:35 But if you consider more than this simplified ontology, then this is no longer a DAG anymore, okay?
00:10:44 So what we often do is a simplified graph, but it also has limited information, especially the relationship we can have here.
00:10:54 Any questions so far?
00:10:56 Any questions so far?
00:10:58 So how can you predict a protein function, right?
00:11:04 What are the inputs you can do, right?
00:11:06 I'm not talking about how you represented the input.
00:11:08 I'm just asking what kind of input, right?
00:11:11 Apparently, we said that there is a sequence, there's structure, right?
00:11:15 How do we represent the sequence?
00:11:16 How do we represent the structure?
00:11:18 That's a different issue.
00:11:19 Like, you observe in the threading how we represent the query sequence or how we represent the template structure, right?
00:11:25 We could use profiles, we could use the secondary structures, and so on.
00:11:29 That's how you represent, right?
00:11:31 But if you ask the type of data, we have the sequence, we have the structure, and there's more.
00:11:37 More than that, okay?
00:11:39 For instance, you can look at the protein-protein interaction networks, okay?
00:11:43 So if you ever, you know, met some new friend and you want to know more about him or her, right?
00:11:53 But his Instagram is not open to everyone, right?
00:11:58 And then you find a mutual friend, maybe.
00:12:01 Then you guess, well, maybe they share similar interests, right?
00:12:05 Right?
00:12:06 So in other words, you may not know a person well, but if you know that person's companions,
00:12:13 then you can probably tell what that person is doing.
00:12:16 So this is a Jewish proverb, okay?
00:12:19 Tell me who your companions are, and I will tell you what you are, okay?
00:12:25 Another kind of data is the protein expression profiles.
00:12:28 So in different environments, you have different genes or proteins of different expressions, okay?
00:12:35 Sometimes you may see that some genes or proteins expressions are having a strong correlation.
00:12:42 They both go up or down together, or one goes up, one goes down.
00:12:46 We'll continue the introductory of the second module, machine learning,
00:12:54 and we are motivating this problem, sorry, this topic with the problem of protein function predictions, okay?
00:13:02 So we talked about what is a protein function, or more specifically,
00:13:10 how can we describe a protein function?
00:13:13 Can someone tell me how we describe a protein function?
00:13:20 Using what?
00:13:24 Two words.
00:13:29 How do you describe a protein function?
00:13:35 Gene ontology, G-O terms, okay?
00:13:40 There are three ontologies, molecular function, biological processes, and cellular component, okay?
00:13:48 So it's about what these individuals do, what is it a part of, and where does that happen?
00:13:54 So these are the ontologies, okay?
00:13:58 All right, so now let's, you know, think about this, you know, you learn a lot of machine learning
00:14:06 courses already, so here I'm just motivating the fact that we are using a lot of data
00:14:15 and integrating them together for the purpose of protein function prediction, right?
00:14:21 So I'm giving you one example, right?
00:14:23 So here, you know, you can jointly embed the input features and the output labels.
00:14:32 So the input is a sequence, which is in a text, and the output is defined as probability over
00:14:48 the gene ontology, which is a graph, right?
00:14:51 So in this sense, what you can do is that you can jointly embed the input sequence and the gene ontology labels, right?
00:15:05 So that after embedding, right, they're sort of close to each other with some proximity,
00:15:13 and then with this learned embedding, you can do this prediction using fully connected layers for the probabilities of the geo terms, right?
00:15:25 So this is just for individual proteins functions,
00:15:31 but you understand that this is just one kind of signal or input protein sequence, right?
00:15:37 There are a lot of other data, like what we described before, including protein-protein interaction networks.
00:15:43 If you think about it, actually, if you only rely on protein sequence to make a prediction
00:15:48 on the molecular function, maybe that makes sense to some extent.
00:15:54 But if you think about the second ontology, which is biological process,
00:15:59 right, it's asking who are you teamed together with, right?
00:16:04 So in what big process are you part of?
00:16:07 So in that case, you definitely want to use additional data, right?
00:16:11 Because this protein-protein interaction network must tell you a lot of information about the biological process, right?
00:16:18 So and then there's a cellular component as well.
00:16:22 So in other words, in general, you would imagine that if you use more data that are more related to the molecular,
00:16:30 sorry, the gene ontologies, you may get better results and you have a rationale about that, right?
00:16:37 Especially because of the other ontologies, other than molecular function, are encoded more than,
00:16:46 are encoded in more than protein sequences.
00:16:49 It's also encoded in the protein interactions and so on, right?
00:16:52 And expressions.
00:16:53 So the quick question for you is that now, okay, here I'm embedding the individual protein sequences
00:17:00 and a constant label matrix, okay?
00:17:05 How do I incorporate the protein-protein interaction data into this framework?
00:17:13 Any ideas?
00:17:15 Again, here I'm embedding the individual protein sequences as text, right?
00:17:21 The additional trick I used is to embed the graph so that the embedding of the sequences and the embedding of the graph labels
00:17:28 are kind of close, right?
00:17:30 In the latent space, right?
00:17:32 You can forget about this for the moment, right?
00:17:35 Because this is a constant branch for an arbitrary protein sequence, right?
00:17:40 Because the label is unknown.
00:17:41 This is a fixed k by k matrix where k is a number of geoterms on the gene ontologies, right?
00:17:50 So if now you want to embed, sorry, incorporate additional data about protein-protein interactions,
00:17:57 what would you do?
00:18:02 Any suggestions?
00:18:02 Any suggestions?
00:18:11 How would you model protein-protein interaction data?
00:18:15 What kind of, sorry, what kind of data modality is this?
00:18:21 And what kind of models are good at learning from such data?
00:18:25 What kind of models are good at learning from such data?
00:18:31 Well, you can represent the protein-protein interaction network as a graph, right?
00:18:38 And what can you use to model graph data, for instance?
00:18:46 There are many choices, right?
00:18:58 For instance, I can pick one of the latest hot topic, graph neural networks, right?
00:19:06 So then on this graph neural network, right, what is each node?
00:19:11 What does each node represent?
00:19:19 The graph is for protein-protein interaction.
00:19:22 So what is each node?
00:19:29 A protein, right?
00:19:33 You follow?
00:19:33 So this individual protein encoder, right, used to be IID in the training set,
00:19:43 can now be correlated with each other because on each node, you have such a sequence embedder,
00:19:54 the same sequence embedder, but with different sequence inputs, right?
00:19:59 So in other words, you are training two things together.
00:20:06 One is how to embed the sequences.
00:20:09 Two is how these interacting proteins affect each other's embeddings, right?
00:20:16 You are learning the text embedding and the graph embedding together, right?
00:20:23 So that's one example.
00:20:24 Does that make sense to you?
00:20:25 Okay, this could be a final project, okay?
00:20:31 Any questions?
00:20:35 Okay.
00:20:36 So let's, again, try to, you know, look at this problem.
00:20:40 It's actually done in a community assessment called CAFA, Critical Assessment of Function Annotation.
00:20:48 There was an old summary which says that the challenge for these methods, you know,
00:20:53 the first of all, you're looking at the half of the methods use data beyond the sequence similarity.
00:21:01 The sequence similarity is the basic thing, right?
00:21:03 So this joint embedding we did was just digging the sequence similarity in the latent space, right?
00:21:08 But still, there are evolutionary relationships, protein structures, protein interactions, and gene expression data, okay?
00:21:15 So the challenge for these methods was finding ways to integrate disparate data sources and properly handle incomplete and noisy data.
00:21:26 Like what I said, right, you cannot just, you know, featurize them and concatenate them and just dump them into a generic neural networks, right?
00:21:34 So most methods on the computational side use a machine learning principle, okay?
00:21:41 So, but they do require experiences in the classification models, learning parameters, feature of the training set, and so on, right?
00:21:50 And among the three categories, performance may be considered accurate for molecular function,
00:21:56 which is mostly determined by protein sequences, but in the case of biological process, overall performance was below, okay?
00:22:05 Okay, so we actually have the CAFA test set, right, accompanied with this paper, so we can compare our performances to the whole community as well.
00:22:19 So, in this sense, let's now think about it, why, right?
00:22:25 Everybody is using machine learning to solve this problem for a reason, right?
00:22:29 Not because machine learning is hot, right?