Lecture 11.2: Introduction to Machine Learning
Recording. Download the text transcript.
Automatic transcript; technical terms and equations may contain recognition errors.
00:00:00 So let's start motivating when and why do we want to use machine learning.
00:00:05 It feels like a no-brainer, but still you want to think about it.
00:00:09 It's not like it's always machine learning.
00:00:11 In this case, why it was a no-brainer for machine learning?
00:00:15 Well, we are studying a very complex systems that is one reality.
00:00:22 And then we'll tell what does this reality entail.
00:00:26 So protein alone is already very complex, as I said.
00:00:31 The rule of the sequence structure function biology is there's no golden rule.
00:00:37 And then if you think about the whole system, actually it consists of a lot of molecules,
00:00:44 including proteins that interact with each other hierarchically.
00:00:49 They form pathways, cells, individuals, and populations, and so on.
00:00:54 So it's a very complex system where the data are generated.
00:01:00 And then what?
00:01:03 So when you want to model some complex systems like this, there are a bunch of challenges.
00:01:09 One is the computational scalability.
00:01:12 So it's one thing to predict a single protein structure and dynamics.
00:01:16 It's another thing about predicting a pair of protein or tens or hundreds of protein interacting
00:01:22 with each other.
00:01:23 And it's another thing to model all these proteins and molecules within the cell towards a function.
00:01:29 So the scalability is one issue that you have to think about for your method.
00:01:35 And the second thing is intrinsic complexity.
00:01:40 Okay?
00:01:41 So think about it.
00:01:42 We already simplified a lot of things, made a lot of assumptions.
00:01:47 When we solve protein structure prediction as an energy minimization problem for using the molecular mechanics,
00:01:56 we already made a lot of assumptions about the independence between the electron and neutrons.
00:02:10 And we can separately model them, right?
00:02:13 The charge-charge interaction, dipole-dipole interaction.
00:02:17 We sort of, you know, put them into different categories of interactions.
00:02:22 And then even that is sophisticated and it's depending on the environment, like the pH, the crowding effects, and so on.
00:02:31 All right?
00:02:32 I remember, was that Heymin who asked me before, like, what was that number on the amino acid?
00:02:37 Two-dimensional graphs, right?
00:02:39 That number was, you know, pKa, which is the critical pH values when the proton will be lost.
00:02:46 Right?
00:02:47 So it's very complicated.
00:02:49 Right?
00:02:50 And the protein function, you have to actually consider the location, where it happens.
00:02:55 Is it a dimer or oligomer?
00:02:57 And is it partner-specific?
00:02:58 So it's very sophisticated.
00:03:00 Just think about yourself.
00:03:01 How do you describe your function, right?
00:03:05 Right?
00:03:06 Right?
00:03:07 So in different contexts, when you are interacting with different people, you are fulfilling different functions.
00:03:12 So, and on top of that, we have the incomplete knowledge.
00:03:16 This very famous, very famous quote from, you can tell me who it's coming from.
00:03:25 There's known unknowns and there's unknown unknowns.
00:03:30 Who's this quote coming from?
00:03:33 Well, we have a war starting today, actually.
00:03:40 And this quote was made years ago when another war started.
00:03:48 Okay, you can find it out.
00:03:54 Okay?
00:03:55 So, in short, the conclusion is that you believe that you can collect more and more data and
00:04:04 understand the system more and more so that you would have a very sophisticated, gigantic
00:04:11 model describing the complex system.
00:04:16 And you are just trying to make sure that happens.
00:04:20 Well, it's very difficult.
00:04:22 Okay?
00:04:23 If you learn control, you know that the best control strategy is not necessarily built upon the precise,
00:04:30 I'm sorry, built upon more detailed description of the system.
00:04:36 Okay?
00:04:37 Sometimes it depends on the resolution of the data.
00:04:39 So, based on this conclusion, what machine learning is trying to do is the following.
00:04:51 Machine learning is extremely powerful when you have a system, a complex system, but you have a lack of laws and details.
00:05:04 Right?
00:05:05 As you said, you don't even know all the components involved, and you don't know how they are
00:05:09 interacting with each other in details.
00:05:12 Or sometimes you have a law, you know, the energy minimum principle, but you don't have the details to calculate the Schrodinger equation in a reasonable time.
00:05:27 Right?
00:05:28 In contrast to the lack of laws and details in this natural system, you have abundance of data.
00:05:36 This data is the paired input-output examples.
00:05:41 So you have a lot of IO pairs.
00:05:43 So this is the system that you know machine learning got to be the way because it's too hard to be modeled precisely following principle,
00:05:53 because there's a lack of principle and details.
00:05:56 But in the meantime, you have abundance of data.
00:05:59 So now let's look at this system.
00:06:01 Right?
00:06:02 This is a natural process.
00:06:03 Right?
00:06:04 Whose input includes X and X prime.
00:06:09 And X and X prime, let's think about this process as a causal process.
00:06:14 Okay?
00:06:15 And X and X prime can result in the output.
00:06:19 Let's call it a Y or F of X and X prime.
00:06:23 But you don't know who X prime are.
00:06:26 Or you have no way to measure X prime.
00:06:30 So all you have are a bunch of X and F of X X prime pairs.
00:06:37 Okay?
00:06:38 So that's what you have.
00:06:39 So based on such data, your goal is to construct an artificial model using some learning algorithm and domain knowledge.
00:06:51 Right?
00:06:52 And you want to make a prediction from X.
00:06:55 Notice that it's from X alone to the prediction Y hat or H.
00:07:01 Right?
00:07:02 And usually you want construct such or train such model to replicate the observed IO relationships from the natural processes.
00:07:14 Once you have such model trained, then you can make these predictions with different X.
00:07:20 So this is what machine learning is good at.
00:07:25 Okay?
00:07:26 So many of you have taken or are taking machine learning courses.
00:07:33 So I'm not going to repeat too many repeated knowledge.
00:07:38 But I just want to remind you that there are a lot of things that are important in machine learning, including theories.
00:07:47 Okay?
00:07:48 And we know already from optimization that theories are really important like convergence, convergence rate, complexity.
00:07:57 They actually help us in the way of developing algorithms.
00:08:02 Because without them, it's hard to guide your development.
00:08:04 Is that right?
00:08:05 Because you can only trial and error.
00:08:07 But that price is something very expensive.
00:08:12 So, you have learned bias variance trade-off, right?
00:08:20 Have you?
00:08:22 Bias variance trade-off?
00:08:25 Yes.
00:08:26 Yes.
00:08:27 Okay.
00:08:28 So this is a very important concept.
00:08:30 And we'll discuss them more in details.
00:08:33 But I believe you have learned and or are learning.
00:08:37 Right?
00:08:38 I know more than two or three of you are taking machine learning courses this semester.
00:08:42 Right?
00:08:43 So there is a very important concept about the complexity
00:08:49 of the model and the generalizability of the model.
00:08:53 So that's what the trade-off is trying to address.
00:08:55 And then knowing that the complexity is important
00:08:59 to the generalizability, then we can regularize
00:09:02 or control the complexity to affect the generalizability.
00:09:08 This will be discussed, and at least repeated
00:09:11 in this lecture, in our courses.
00:09:14 Then there are a lot of other things, right,
00:09:16 including the time complexity, including learnability, right?
00:09:21 I believe that PAC learnability is definitely mentioned
00:09:25 or will be mentioned in machine learning with networks.
00:09:27 So probably approximately correct PAC learnability.
00:09:35 So these are the theories involved.
00:09:37 And I have very incomplete list of machine learning algorithms
00:09:42 because here these are just the focus on supervised
00:09:45 versus unsupervised.
00:09:47 But you can also think about the different modalities, right,
00:09:52 involved.
00:09:53 And, you know, here I just put everything under deep learning,
00:09:57 but you know that it's way more than that.
00:09:59 So luckily we have a whole third module, right, almost 50% of the
00:10:04 lectures on the deep learning part, okay?
00:10:08 All right.
00:10:09 Any questions so far as we start on this part?
00:10:15 Okay, let's have a formal setup, okay?
00:10:18 So as I said, what we have is a list of paired X and Y's input and outputs, okay?
00:10:28 So let's say the input data is drawn from a space X and the output, also called label or target,
00:10:37 is drawn from a space Y, okay?
00:10:40 So X is also called the feature space, right?
00:10:43 Sometimes you can roughly say that each dimension is a feature,
00:10:47 but it also depends because sometimes people think that X, each dimension is just the attribute.
00:10:52 So what feature means is something that you extracted from the attributes.
00:10:59 So those are just the names.
00:11:01 So if the space, the target or the label space, output space is finite,
00:11:10 then we're talking about a classification problem, right?
00:11:13 You have finite number of labels, but if it's continuous,
00:11:16 we're talking about a regression problem.
00:11:18 And the unknown function we're going to try to learn is from X to Y.
00:11:24 So it's a mapping between the two spaces.
00:11:27 And as I said earlier on, we are given a set of IO pairs.
00:11:33 So the input is Xi, it's bold-faced, meaning that it's a vector.
00:11:37 And the output is Y sub i, meaning that it's a scalar, it's not bold-faced.
00:11:42 So for now we just consider a scalar variable, sorry, a scalar label.
00:11:48 And we have N such so-called examples, okay?
00:11:54 Paired input-output relationships, okay?
00:11:57 So imagine that you are a hiker, you are a tea drinker, you notice that, right?
00:12:04 The time needed to boil water to make your tea is different when you are at home
00:12:13 or when you are at the top of the mountains, right?
00:12:15 Though this, right, you may not know the details of the physics yet for now, okay?
00:12:20 I'm assuming, but of course in this case, in the physics is not that sophisticated.
00:12:24 So you don't need the machine learning to tell that.
00:12:26 But anyway, let's say this was before the discovery of that physics.
00:12:31 So what you can do is, you know, you can just look at different locations
00:12:36 and measure the pressure at that location, that's the input,
00:12:41 and also measure the boiling temperature of boiling point at that location, that's output.
00:12:49 Then you can find out that there's some, yeah, almost linear.
00:12:53 You don't need the machine learning, but your brain is already doing machine, right?
00:12:57 Human learning.
00:12:58 There's a model you think that, oh, it looks like linear, right?
00:13:02 Believe it or not, our brains are regularizing the complexity of the models with our own priors, okay?
00:13:09 So here we actually think that a linear model probably would suffice, okay?
00:13:16 But I think in machine learning, we are doing things in a more clear way.
00:13:21 We don't know fully how brains function, okay?
00:13:25 Okay, so this is a formal setup.
00:13:27 And again, you have learned all this before.
00:13:31 I don't want to talk too much about it.
00:13:33 We are talking about supervised versus unsupervised.
00:13:37 And then in the case of the supervised, depending on the label,
00:13:40 we have classification versus regression.
00:13:42 And then in unsupervised, actually, I list some examples.
00:13:47 But what is not included included this unsupervised learning and self-supervised learning in deep learning.
00:13:55 So those will be partially covered in the deep learning module, okay?
00:14:01 Okay, so let's see.
00:14:06 Let me ask you a question.
00:14:08 What kind of machine learning problem are we dealing with when we try to derive statistical potentials?
00:14:17 Remember, we are using statistical potentials to replace the physics-based energy models like force fields
00:14:25 so that we can optimize that function to find the best structure prediction.
00:14:30 So what kind of learning algorithm did we use to derive the statistical potentials?
00:14:36 Probability of the sequence, sorry, probability of the structure given the sequence?
00:14:49 Well, what feature do you have for each structure?
00:14:55 Some kind of embedding of the structure, okay?
00:15:00 But what label do we have?
00:15:03 The label is how close this structure is compared to the native structure, which is the ground truth, right?
00:15:11 Let's call it RMSD, root mean square deviation.
00:15:16 Remember, when we compare the structure similarity, we use the root mean square deviation, okay?
00:15:21 So if I use the root mean square deviation as a label and learn a function to reconstruct such relationship,
00:15:28 I am doing a regression problem, right?
00:15:32 But I could also classify this, you know, binarized label based on the root mean square deviation, right?
00:15:39 That's what David Baker did in the earlier Rosetta.
00:15:42 So they look at the root mean square deviation, if it's less something, they say it's near native.
00:15:47 If it's bigger than something, it's, you know, not native.
00:15:52 And then he used the logistical regression for the classification problem, right?
00:15:57 So binary classification.
00:15:58 So it depends, okay?
00:16:00 Now, what kind of learning are we talking about for protein function prediction?
00:16:05 Is it supervised, unsupervised?
00:16:11 Is it classification or regression?
00:16:14 Huh?
00:16:19 Unsupervised.
00:16:20 Supervised.
00:16:21 Okay, why?
00:16:23 Why unsupervised?
00:16:25 There's a huge amount of data that is not labeled.
00:16:42 Okay.
00:16:44 I see what you mean.
00:16:47 Okay, let me explain a little bit.
00:16:49 Sorry for the confusion.
00:16:50 When I put no labels down there below unsupervised learning, what I meant was not that what you want
00:16:58 to predict for does not have labels.
00:17:01 And what I meant was that in the process of making a prediction or learning a model, there's
00:17:11 no label data you utilize.
00:17:13 You see what I mean?
00:17:15 So when you construct that model for machine learning, there's no Y.
00:17:20 There's only X.
00:17:21 I'll give you one example, right?
00:17:24 For instance, I look at, you know, the Google Earth image of the world at night, right?
00:17:37 Then I see a lot of lights, right?
00:17:40 And then the size of, I'm sorry, the size of the light, I'm sorry, the intensity of the light
00:17:45 is like how much human population is there, right?
00:17:50 Activities are there.
00:17:51 Sorry, activities.
00:17:52 Now, not necessarily population because there are populations who don't have a lot of access
00:17:56 to electricity, right?
00:17:57 And then the size is sort of showing the metropolitan area size, right?
00:18:02 So in that case, my X are what?
00:18:06 Locations and intensities of these lights, right?
00:18:10 And I do clustering and I found the big cities, right?
00:18:14 So I didn't use any Y.
00:18:16 You see what I mean?
00:18:18 So in that case, we say it's unsupervised, okay?
00:18:22 So is that clear?
00:18:27 Okay.
00:18:28 So I'm going to come back to you, okay?
00:18:31 So let's ask Doyoung again.
00:18:34 Why do you think it's a supervised learning problem?
00:18:37 Good.
00:18:38 So you're saying that, oh, we do have some percentage of protein sequences whose functions
00:18:53 are already labeled, right?
00:18:55 Okay.
00:18:56 Then is that a classification problem or regression problem?
00:19:00 What are the labels?
00:19:01 Is the label space discrete or continuous?
00:19:14 Discrete.
00:19:15 Discrete, because we are looking at geo terms on the gene ontology, right?
00:19:20 So in that sense, you can say it's discrete or classification.
00:19:24 But you can also, just like a logistic regression, right?
00:19:27 It's for classification, but it calculates the continuous probabilities in each class.
00:19:32 So in this case, you can also do this multi-class classification, right?
00:19:36 Okay.
00:19:37 But now, thank you for the answer.
00:19:40 Now let's come back to what Ryan said.
00:19:43 Actually, it's also correct.
00:19:46 Because there are a lot of data for protein sequences without labels yet.
00:19:52 So does that mean we cannot use them?
00:19:57 Can we still use them?
00:20:02 How do we use them?
00:20:05 Can you cluster proteins based on their gene expression?
00:20:22 Then you assume that genes in the same cluster should have the same or similar function.
00:20:33 Then that's also the approach, right?
00:20:36 So in reality, you could combine these things together, okay?
00:20:39 So it depends on your formulation.
00:20:42 It's not a fixed scenario.
00:20:44 So there's no correct, single correct answer for this question.
00:20:48 I hope this shows you the, you know, this is part of your design.
00:20:53 Okay.
00:20:56 All right.
00:20:57 So as I said, protein function prediction, it can be a supervised classification problem,
00:21:02 where the input are the features and the output are labels, which are the leaf terms of the geo terms.
00:21:08 So it's the most detailed gene ontology.
00:21:12 So it's supervised classification.
00:21:14 It can be supervised regression, right?
00:21:17 The labels is likelihood between 0 and 1 for each leaf.
00:21:22 But it's not really regression.
00:21:24 It's still classification.
00:21:25 Okay.
00:21:26 Okay.
00:21:27 So just, but the, the, the, what do you predict?
00:21:30 It can be different, right?
00:21:32 Um, another one is unsupervised classification.
00:21:36 You can group related genes based on the similarity, right?
00:21:40 And this, you can, this also can be, uh, semi-supervised.
00:21:45 In other words, you can use, uh, unlabeled test set, including, so, so, so, so you can also use unlabeled test set.
00:21:55 In other words, for the test set, you don't know their labels, but you can use their features.
00:22:00 And then you can use the semi-supervised learning, right?
00:22:03 You, you have learned what semi-supervised learning means, right?
00:22:06 Okay.
00:22:07 So, all right, let's, uh, you know, I don't have to say too much about it, but, uh, in the case of the supervised learning, right,
00:22:21 what you, you really do is, uh, you process the data and, uh, split the data into the training and test set.
00:22:28 You select the algorithms based on the kind of problem you deal with.
00:22:32 So, uh, you know, you're doing the model, uh, finding the parameters based on the training set.
00:22:44 And then you were evaluate over the validation set, uh, what they say test here, I think should be validation or development.
00:22:51 And then you find out which hyperparameter is the best and the, you know, the optimal parameters also associated with that, um, best hyperparameters.
00:23:02 And then, uh, eventually you find a best model or ensemble of models and, uh, you can, uh, deploy that.
00:23:11 Okay.
00:23:12 So when you choose an algorithm, you can, based on, uh, evaluating the performances, uh, cross-validation, leave one out.
00:23:21 Uh, if you are taking pattern recognition course, you will learn about more details and theoretical, uh, aspects, especially when the sample size is small,
00:23:33 whether cross-validation or leave one out is still a good idea.
00:23:37 So you will learn a lot, okay, from, uh, Professor Baganato in that course.
00:23:42 And then, uh, you could also do the statistical comparisons, right?
00:23:46 For instance, uh, rock curve for classification.
00:23:49 We're gonna remind you again for that.
00:23:51 Okay.
00:23:52 Okay.
00:23:53 So, uh, how to represent the input.
00:23:57 I said in the, you know, I asked this multiple times, uh, and, uh, we have seen an example in the threading, uh, problems.
00:24:06 And we also asked you a couple of times about, uh, how do you represent the protein structure for deep learning model, right?
00:24:13 So, and that actually determines your, uh, model choice, right?
00:24:18 Do you represent them as, um, point clouds?
00:24:23 Do you, uh, represent them as a rigid body?
00:24:27 So it's just a volume.
00:24:29 Uh, do you represent them as just a surface, right?
00:24:34 Manifold?
00:24:35 Uh, you could have different choices, right?
00:24:37 Indeed, they are being represented in different ways, right?
00:24:40 Or you represent them as graph with residues interacting with each other on the, on the nodes, right?
00:24:46 So we learned this before that, uh, you could represent, um, so I know that we talk about, uh, the fact that we can use end-to-end models, right?
00:24:57 And similarly, you know, feature engineering can be reduced.
00:25:03 But honestly, um, feature engineering is still very important.
00:25:08 Uh, it's important to have the domain meaningful features, okay?
00:25:14 Because if you want your model, naive model, to learn such knowledge directly from abundant data, it's very difficult.
00:25:25 Okay, give you one simple example.
00:25:28 If you don't tell the model in a way that, you know, this, uh, label is, uh, invariant to the translation of the object, X.
00:25:43 For instance, however I translate the image, or the cat in the image, it's still a cat.
00:25:50 Sorry, did I say translate?
00:25:52 Rotate, sorry.
00:25:54 Right?
00:25:55 So, convolutional neural networks can achieve the translational invariance.
00:26:00 That's not a problem, right?
00:26:02 So we'll talk about this in more details.
00:26:04 But it doesn't know rotational invariance, does it?
00:26:07 Right?
00:26:09 So, so how do people usually teach a convolutional neural networks about the rotation invariance?
00:26:17 Usually they just have to let this model learn from a lot of data.
00:26:23 For each X, Y pair, you randomly rotate X many times and so-called augment the data set, training set.
00:26:32 Right?
00:26:33 Then the model hopefully sees that, oh, I have so many examples that their X are different but rotated relative to each other and their Ys are the same.
00:26:41 Okay?
00:26:42 But still, it's a very low efficiency.
00:26:47 Okay?
00:26:48 And there's no guarantee that it's, you know, not to mention that you're making this learning process extremely painful.
00:26:56 Okay?
00:26:57 And so for that sense, you do have to do some feature engineering sometimes.
00:27:04 And that does matter.
00:27:05 Okay?
00:27:06 So we talked about this before.
00:27:07 You can represent them in sequence, profile, and structure.
00:27:10 But even for sequence, right?
00:27:12 So there are a lot of things than just doing a one hot.
00:27:16 Right?
00:27:17 In homework three, you will learn something about it.
00:27:20 You will see how you choose your input representation really matters.
00:27:26 And if you just tell them that, oh, these are just the 20 different letters, then it's extremely difficult for the machine learning model to figure out how to assign a sequence to a label.
00:27:40 Because it has no idea that this letter actually corresponds to a hydrophobic small residue.
00:27:47 And that letter is also a hydrophobic but a bulkier residue.
00:27:53 It has no concept at all.
00:27:56 Imagine that you just, the first time you learn the alphabet, if you just look at these 26 letters, right, what can you say?
00:28:06 The only thing you can say is they are different.
00:28:08 Okay?
00:28:09 You have no knowledge.
00:28:11 And how many examples after that did you need to finally have a concept, right?
00:28:18 But in that case, we still have rules, right?
00:28:21 So it's important to represent the inputs and incorporate the domain knowledge.
00:28:28 So the domain knowledge is in your input.
00:28:30 I'm sorry.
00:28:31 It's first of all in your design of the, or formulation of the problem.
00:28:36 Then it's in your input.
00:28:38 Then it's also in your model architecture accordingly.
00:28:42 So that's where you can introduce the domain knowledge.
00:28:48 And lastly, what I didn't say was the last function, the objective function, right?
00:28:52 That's one of the most important things for the optimization problems.
00:28:56 You could represent inputs differently than just using features.
00:29:06 Okay?
00:29:07 Instead of just representing each x with a lot of features, you can actually also use something called kernels.
00:29:16 Okay?
00:29:17 Kernels is, you know, you can roughly think of it as a distance.
00:29:23 Okay?
00:29:24 So in other words, I don't tell you what is the coordinate of each sample x in the space.
00:29:33 But I do tell you what's the distance or similarity or dissimilarity between two, between a pair of examples, xi and xj.
00:29:44 You don't know where, what are the coordinates or the features of xi and xj.
00:29:49 But you do know the distance or similarity or dissimilarity between xi and xj.
00:29:54 Is that clear?
00:29:55 So that's what the kernel representation means.
00:29:58 Okay?
00:29:59 And this could be a Euclidean distance in the protein feature space, but this can be a bad choice.
00:30:07 Right?
00:30:08 Because you stack all the different features into a space and then different dimensions can be different types of variables and the scales are different.
00:30:20 If you just use the Euclidean distance in that space, it doesn't make any sense.
00:30:25 Right?
00:30:26 But in your applications, very often, people have some domain knowledge defined distance concept.
00:30:35 Right?
00:30:36 Like in our structure similarity, we know how to measure structure similarity.
00:30:40 Right?
00:30:41 So in other kind of analysis, you have some idea about what similarity means.
00:30:46 Right?
00:30:47 So you want to teach that to the machine learning model.
00:30:50 Right?
00:30:51 Then this, you know, kernel representation is another way.
00:30:55 We'll actually see the kernel representation in the support vector machines again.
00:31:00 Okay?
00:31:01 So, and I want you to carry that powerful concept when we review the kernel methods.
00:31:09 And how we carry that on all the way to deep learning, that's another very active line of research.
00:31:16 Okay.
00:31:17 So, yes, as I said, these kernel methods are very often used.
00:31:23 Like, k nearest neighbors.
00:31:25 Right?
00:31:26 You don't care about the features.
00:31:27 All you care about is who are my neighbors.
00:31:29 Right?
00:31:30 And then the support vector machine as well, actually.
00:31:33 It's a kernel method.
00:31:34 Okay?
00:31:35 All right.
00:31:37 So, now let's finish this motivation part about why we want to do machine learning.
00:31:43 Right?
00:31:44 Generally, if there's a lack of general laws or specific details for a very complex system,
00:31:52 but for that system you have abundance of I.O. data, then it's a perfect scenario for machine learning.
00:32:02 Because you can construct an artificial model to reconstruct the I.O. relationship and try to make generalization.
00:32:10 So, that's when you use machine learning.
00:32:13 Okay?
00:32:14 But you don't use machine learning to discover a principle or, you know, F equal to M.A. from data.
00:32:22 Because you already know that, you just want to incorporate that directly into your model.
00:32:26 Okay?
00:32:27 And then we talked about a feature, label, training set, test set, different kind of learning scenarios,
00:32:33 and what theories and algorithms might be involved, mostly in the shallow, because the deep learning part will be in different modules.
00:32:40 [Uncertain recognition] Right?