Lecture 11.1: Protein Function Prediction

Recording. Download the text transcript.

Automatic transcript; technical terms and equations may contain recognition errors.

00:00:00 So this will be the end of the optimization module.

00:00:03 Any questions so far?

00:00:06 Okay, now I'm gonna start the next module

00:00:13 using a different problem to motivate this formulation.

00:00:19 Okay, so we're talking about machine learning,

00:00:21 but you know optimization and machine learning

00:00:23 are two sides of a coin.

00:00:25 So I'm gonna start with the protein function prediction.

00:00:29 So far we're doing sequence and structure,

00:00:31 but don't forget function, okay?

00:00:35 So where we are, we are right here, okay?

00:00:40 We're gonna use this to explain the basic concepts,

00:00:44 but I think many of you already know machine learning already,

00:00:48 so I'm gonna go through this part really quickly,

00:00:51 and sometimes I will just directly ask you,

00:00:55 you know, what do you recollect for those concepts, okay?

00:00:59 And at the end of today's lecture,

00:01:01 I will ask you a question about support vector machine,

00:01:04 so please don't let me forget about it.

00:01:07 Please remind me, okay?

00:01:10 Okay, so, motivation, right?

00:01:13 So we're now moving to the next question, right?

00:01:17 Protein function prediction.

00:01:18 So why, what, and how?

00:01:21 Okay, and then how this motivates,

00:01:24 how the challenge motivated machine learning models, okay?

00:01:28 So why, we know that there is a gap between

00:01:32 the sequentially determined and the structurally determined

00:01:35 proteins, right?

00:01:36 But also the gap between the sequenced determined

00:01:40 and the structurally annotated, sorry,

00:01:42 functionally annotated proteins are also wide, okay?

00:01:45 For instance, this is the green one, Swiss prot, okay?

00:01:50 This is the manually annotated protein sequences, okay?

00:01:54 Those sequences are already manually annotated

00:01:57 with the function, okay?

00:01:59 But there are way more sequences, right?

00:02:01 In tremble, which include a lot of sequences

00:02:03 that haven't been annotated or just annotated automatically, okay?

00:02:06 So these three data cells are getting bigger and bigger.

00:02:13 Protein sequences that have been structurally determined,

00:02:17 protein sequences that have been functionally annotated,

00:02:21 and protein sequences that have been collected, okay?

00:02:26 So, and if you look at this,

00:02:30 this is the same thing.

00:02:33 This genome-wide protein function prediction,

00:02:38 this is old data, but still,

00:02:40 more than 40% of the known sequences, right?

00:02:45 In our species, even after you exhaust

00:02:48 the homology-based method for annotation,

00:02:51 you still end up having like 40%, okay?

00:02:55 40% of the molecular functions unknown.

00:02:59 So now, the question is,

00:03:02 how do we do that?

00:03:04 Again, why is this a computational problem?

00:03:06 You told me that sequence is 1D,

00:03:08 structure is 3D, I understand.

00:03:11 But how do you even describe a function?

00:03:15 I don't even know what's the dimension, right?

00:03:17 So like what we did before,

00:03:19 we looked at the sequence structure relationship before,

00:03:23 and ended up knowing that there are some,

00:03:27 you know, heuristic rules,

00:03:28 then we can look at the 30% as a cutoff, right?

00:03:32 And figure out whether we are in the safe zone or not,

00:03:34 right?

00:03:36 Here, I'm gonna do something similar,

00:03:38 but we're gonna, you know, talk about more about it.

00:03:41 So sequence, so in general, there's no golden rule, okay?

00:03:45 But there are some, you know, heuristics,

00:03:47 like we said before, safe zone is 30%, okay?

00:03:51 And we knew that the structure is more conserved than sequence,

00:03:54 right?

00:03:55 Like, this can be explained by this bullet points

00:03:58 that we have learned before, okay?

00:04:02 And it's like the one versus 100, okay?

00:04:07 Then what about the sequence versus function?

00:04:09 Well, the safe zone is now at about 40%.

00:04:13 If the identity is about 40%,

00:04:16 then the sequences tended to have similar structures

00:04:20 and the functions, okay?

00:04:22 Again, it's a heuristic.

00:04:25 What about the structure versus function?

00:04:28 Well, it's more complicated and less well known,

00:04:31 but because one issue is that I cannot say that similar structures

00:04:35 give you similar functions,

00:04:37 because I haven't described the functions yet.

00:04:39 One structure can have multiple functions, right?

00:04:44 Right?

00:04:45 So this is another issue,

00:04:49 and also your data might be biased because, look,

00:04:53 all the protein sequences that have been structured determined

00:04:55 are already biased.

00:04:57 Those structures are structures that the current technology,

00:05:01 like NMR, X-ray can collect, right?

00:05:05 So that data set is already biased,

00:05:08 so I cannot really derive my conclusion from such small data set,

00:05:12 right?

00:05:14 So let me give you one example, okay?

00:05:17 So this is one structure, multiple functions, right?

00:05:21 So this protein is an enzyme.

00:05:24 It's called the glucose 6-phosphate isomerase.

00:05:28 When you see an ASE at the end,

00:05:30 you know that it's kind of an enzyme

00:05:32 that makes some chemical reaction much faster.

00:05:36 And this enzyme is basically converting the glucose 6-phosphate substrate

00:05:43 to fructose 6-phosphate, which is a product.

00:05:48 This is what this enzyme does, but it's just the one function, okay?

00:05:53 So if it's forming a dimer, this is a dimer, okay?

00:05:57 You have this one monomer in the front, one monomer in the back, right?

00:06:02 It's symmetric.

00:06:04 So this dimer in the cell is accelerating this chemical reaction, okay?

00:06:13 But it could be a monomer outside of the cell, right?

00:06:18 Which act as neuroleukins, cytokines, and so on.

00:06:22 So they could have different functions, okay?

00:06:25 So now you see there's a problem, right?

00:06:28 Functions, first of all, there's no natural dimensionalities to begin with.

00:06:33 Secondly, function is not just sequence and the structure determined.

00:06:38 It's also context dependent, right?

00:06:42 If it's in the cell or it's outside of the cell,

00:06:45 if it's a dimer versus a monomer, right?

00:06:47 Or, you know, the function could be what partners or protein interactions you form, right?

00:06:55 So that's a complication.

00:06:57 So then how you define functions?

00:07:00 The way you define protein functions,

00:07:03 how many of you have heard of this gene ontology?

00:07:07 Good.

00:07:10 So it's okay, I'm just trying to figure out.

00:07:12 So gene ontology is basically consisting of three ontologies.

00:07:19 Molecular function, biological process, and cellular component, okay?

00:07:25 Molecular function says, what does this protein do?

00:07:30 Biological processes ask, what processes is it involved in?

00:07:36 And the cellular component asks, where does that happen?

00:07:39 Okay, so this is why we have three ontologies, okay?

00:07:44 The reason that we call it the gene ontologies is because proteins are gene products, right?

00:07:51 Right, remember, it's translated.

00:07:54 So if you need to know what ontology means, we have a lot of ontologies.

00:07:58 It's a formal representation of a body of knowledge within a given domain, okay?

00:08:04 So there are a lot of, it's describing a lot of relationships between classes or terms, okay?

00:08:11 Okay, here we have three ontologies, right?

00:08:15 Okay, as I said, cellular component, biological process, and the molecular functions, okay?

00:08:20 This is one of the earlier versions, but you can go to geneontology.org,

00:08:24 and you can find out that in each ontology we have certain number of nodes,

00:08:31 and each node is one specific molecular function, for instance,

00:08:38 and each node may correspond to multiple proteins.

00:08:43 In other words, multiple proteins may all reside or share that function node, okay?

00:08:50 So, for instance, right, this is a simple illustration.

00:08:58 We focus on the biological process rather than these other two, okay?

00:09:05 Let's start with the biological process, then we basically see that there are a lot of nodes

00:09:10 and a lot of arcs or edges, right?

00:09:13 So there are actually many different kind of relationships,

00:09:17 relationships, but in general, as you start from, now this is like an upside down tree, okay?

00:09:25 If we only consider certain relationships.

00:09:28 So on the top, you have the root.

00:09:31 As you go down to the leaf, you see that if you follow a path,

00:09:39 the function is getting more and more specific, isn't it?

00:09:44 Isn't it?

00:09:45 So, for instance, here is a regulation of biological process.

00:09:50 This is a positive regulation, right?

00:09:53 And for in this case, pigmentation, and this is pigmentation during development, and so on, right?

00:09:59 So it's getting more and more specific.

00:10:01 But it's not general because it depends on what kind of edges you consider, okay?

00:10:06 So it's like an instance, right?

00:10:11 So in this case, pigmentation is a biological process.

00:10:15 Pigmentation during development is a pigmentation.

00:10:18 So I means is a, but there are other kind of relationships, okay?

00:10:23 If you only consider is a, this relationship, then this graph can be roughly regarded as a DAG,

00:10:33 a directed acyclic graph.

00:10:35 But if you consider more than this simplified ontology, then this is no longer a DAG anymore, okay?

00:10:44 So what we often do is a simplified graph, but it also has limited information, especially the relationship we can have here.

00:10:54 Any questions so far?

00:10:56 Any questions so far?

00:10:58 So how can you predict a protein function, right?

00:11:04 What are the inputs you can do, right?

00:11:06 I'm not talking about how you represented the input.

00:11:08 I'm just asking what kind of input, right?

00:11:11 Apparently, we said that there is a sequence, there's structure, right?

00:11:15 How do we represent the sequence?

00:11:16 How do we represent the structure?

00:11:18 That's a different issue.

00:11:19 Like, you observe in the threading how we represent the query sequence or how we represent the template structure, right?

00:11:25 We could use profiles, we could use the secondary structures, and so on.

00:11:29 That's how you represent, right?

00:11:31 But if you ask the type of data, we have the sequence, we have the structure, and there's more.

00:11:37 More than that, okay?

00:11:39 For instance, you can look at the protein-protein interaction networks, okay?

00:11:43 So if you ever, you know, met some new friend and you want to know more about him or her, right?

00:11:53 But his Instagram is not open to everyone, right?

00:11:58 And then you find a mutual friend, maybe.

00:12:01 Then you guess, well, maybe they share similar interests, right?

00:12:05 Right?

00:12:06 So in other words, you may not know a person well, but if you know that person's companions,

00:12:13 then you can probably tell what that person is doing.

00:12:16 So this is a Jewish proverb, okay?

00:12:19 Tell me who your companions are, and I will tell you what you are, okay?

00:12:25 Another kind of data is the protein expression profiles.

00:12:28 So in different environments, you have different genes or proteins of different expressions, okay?

00:12:35 Sometimes you may see that some genes or proteins expressions are having a strong correlation.

00:12:42 They both go up or down together, or one goes up, one goes down.

00:12:46 We'll continue the introductory of the second module, machine learning,

00:12:54 and we are motivating this problem, sorry, this topic with the problem of protein function predictions, okay?

00:13:02 So we talked about what is a protein function, or more specifically,

00:13:10 how can we describe a protein function?

00:13:13 Can someone tell me how we describe a protein function?

00:13:20 Using what?

00:13:24 Two words.

00:13:29 How do you describe a protein function?

00:13:35 Gene ontology, G-O terms, okay?

00:13:40 There are three ontologies, molecular function, biological processes, and cellular component, okay?

00:13:48 So it's about what these individuals do, what is it a part of, and where does that happen?

00:13:54 So these are the ontologies, okay?

00:13:58 All right, so now let's, you know, think about this, you know, you learn a lot of machine learning

00:14:06 courses already, so here I'm just motivating the fact that we are using a lot of data

00:14:15 and integrating them together for the purpose of protein function prediction, right?

00:14:21 So I'm giving you one example, right?

00:14:23 So here, you know, you can jointly embed the input features and the output labels.

00:14:32 So the input is a sequence, which is in a text, and the output is defined as probability over

00:14:48 the gene ontology, which is a graph, right?

00:14:51 So in this sense, what you can do is that you can jointly embed the input sequence and the gene ontology labels, right?

00:15:05 So that after embedding, right, they're sort of close to each other with some proximity,

00:15:13 and then with this learned embedding, you can do this prediction using fully connected layers for the probabilities of the geo terms, right?

00:15:25 So this is just for individual proteins functions,

00:15:31 but you understand that this is just one kind of signal or input protein sequence, right?

00:15:37 There are a lot of other data, like what we described before, including protein-protein interaction networks.

00:15:43 If you think about it, actually, if you only rely on protein sequence to make a prediction

00:15:48 on the molecular function, maybe that makes sense to some extent.

00:15:54 But if you think about the second ontology, which is biological process,

00:15:59 right, it's asking who are you teamed together with, right?

00:16:04 So in what big process are you part of?

00:16:07 So in that case, you definitely want to use additional data, right?

00:16:11 Because this protein-protein interaction network must tell you a lot of information about the biological process, right?

00:16:18 So and then there's a cellular component as well.

00:16:22 So in other words, in general, you would imagine that if you use more data that are more related to the molecular,

00:16:30 sorry, the gene ontologies, you may get better results and you have a rationale about that, right?

00:16:37 Especially because of the other ontologies, other than molecular function, are encoded more than,

00:16:46 are encoded in more than protein sequences.

00:16:49 It's also encoded in the protein interactions and so on, right?

00:16:52 And expressions.

00:16:53 So the quick question for you is that now, okay, here I'm embedding the individual protein sequences

00:17:00 and a constant label matrix, okay?

00:17:05 How do I incorporate the protein-protein interaction data into this framework?

00:17:13 Any ideas?

00:17:15 Again, here I'm embedding the individual protein sequences as text, right?

00:17:21 The additional trick I used is to embed the graph so that the embedding of the sequences and the embedding of the graph labels

00:17:28 are kind of close, right?

00:17:30 In the latent space, right?

00:17:32 You can forget about this for the moment, right?

00:17:35 Because this is a constant branch for an arbitrary protein sequence, right?

00:17:40 Because the label is unknown.

00:17:41 This is a fixed k by k matrix where k is a number of geoterms on the gene ontologies, right?

00:17:50 So if now you want to embed, sorry, incorporate additional data about protein-protein interactions,

00:17:57 what would you do?

00:18:02 Any suggestions?

00:18:02 Any suggestions?

00:18:11 How would you model protein-protein interaction data?

00:18:15 What kind of, sorry, what kind of data modality is this?

00:18:21 And what kind of models are good at learning from such data?

00:18:25 What kind of models are good at learning from such data?

00:18:31 Well, you can represent the protein-protein interaction network as a graph, right?

00:18:38 And what can you use to model graph data, for instance?

00:18:46 There are many choices, right?

00:18:58 For instance, I can pick one of the latest hot topic, graph neural networks, right?

00:19:06 So then on this graph neural network, right, what is each node?

00:19:11 What does each node represent?

00:19:19 The graph is for protein-protein interaction.

00:19:22 So what is each node?

00:19:29 A protein, right?

00:19:33 You follow?

00:19:33 So this individual protein encoder, right, used to be IID in the training set,

00:19:43 can now be correlated with each other because on each node, you have such a sequence embedder,

00:19:54 the same sequence embedder, but with different sequence inputs, right?

00:19:59 So in other words, you are training two things together.

00:20:06 One is how to embed the sequences.

00:20:09 Two is how these interacting proteins affect each other's embeddings, right?

00:20:16 You are learning the text embedding and the graph embedding together, right?

00:20:23 So that's one example.

00:20:24 Does that make sense to you?

00:20:25 Okay, this could be a final project, okay?

00:20:31 Any questions?

00:20:35 Okay.

00:20:36 So let's, again, try to, you know, look at this problem.

00:20:40 It's actually done in a community assessment called CAFA, Critical Assessment of Function Annotation.

00:20:48 There was an old summary which says that the challenge for these methods, you know,

00:20:53 the first of all, you're looking at the half of the methods use data beyond the sequence similarity.

00:21:01 The sequence similarity is the basic thing, right?

00:21:03 So this joint embedding we did was just digging the sequence similarity in the latent space, right?

00:21:08 But still, there are evolutionary relationships, protein structures, protein interactions, and gene expression data, okay?

00:21:15 So the challenge for these methods was finding ways to integrate disparate data sources and properly handle incomplete and noisy data.

00:21:26 Like what I said, right, you cannot just, you know, featurize them and concatenate them and just dump them into a generic neural networks, right?

00:21:34 So most methods on the computational side use a machine learning principle, okay?

00:21:41 So, but they do require experiences in the classification models, learning parameters, feature of the training set, and so on, right?

00:21:50 And among the three categories, performance may be considered accurate for molecular function,

00:21:56 which is mostly determined by protein sequences, but in the case of biological process, overall performance was below, okay?

00:22:05 Okay, so we actually have the CAFA test set, right, accompanied with this paper, so we can compare our performances to the whole community as well.

00:22:19 So, in this sense, let's now think about it, why, right?

00:22:25 Everybody is using machine learning to solve this problem for a reason, right?

00:22:29 Not because machine learning is hot, right?