Lecture 11.2: Introduction to Machine Learning

Recording. Download the text transcript.

Automatic transcript; technical terms and equations may contain recognition errors.

00:00:00 So let's start motivating when and why do we want to use machine learning.

00:00:05 It feels like a no-brainer, but still you want to think about it.

00:00:09 It's not like it's always machine learning.

00:00:11 In this case, why it was a no-brainer for machine learning?

00:00:15 Well, we are studying a very complex systems that is one reality.

00:00:22 And then we'll tell what does this reality entail.

00:00:26 So protein alone is already very complex, as I said.

00:00:31 The rule of the sequence structure function biology is there's no golden rule.

00:00:37 And then if you think about the whole system, actually it consists of a lot of molecules,

00:00:44 including proteins that interact with each other hierarchically.

00:00:49 They form pathways, cells, individuals, and populations, and so on.

00:00:54 So it's a very complex system where the data are generated.

00:01:00 And then what?

00:01:03 So when you want to model some complex systems like this, there are a bunch of challenges.

00:01:09 One is the computational scalability.

00:01:12 So it's one thing to predict a single protein structure and dynamics.

00:01:16 It's another thing about predicting a pair of protein or tens or hundreds of protein interacting

00:01:22 with each other.

00:01:23 And it's another thing to model all these proteins and molecules within the cell towards a function.

00:01:29 So the scalability is one issue that you have to think about for your method.

00:01:35 And the second thing is intrinsic complexity.

00:01:40 Okay?

00:01:41 So think about it.

00:01:42 We already simplified a lot of things, made a lot of assumptions.

00:01:47 When we solve protein structure prediction as an energy minimization problem for using the molecular mechanics,

00:01:56 we already made a lot of assumptions about the independence between the electron and neutrons.

00:02:10 And we can separately model them, right?

00:02:13 The charge-charge interaction, dipole-dipole interaction.

00:02:17 We sort of, you know, put them into different categories of interactions.

00:02:22 And then even that is sophisticated and it's depending on the environment, like the pH, the crowding effects, and so on.

00:02:31 All right?

00:02:32 I remember, was that Heymin who asked me before, like, what was that number on the amino acid?

00:02:37 Two-dimensional graphs, right?

00:02:39 That number was, you know, pKa, which is the critical pH values when the proton will be lost.

00:02:46 Right?

00:02:47 So it's very complicated.

00:02:49 Right?

00:02:50 And the protein function, you have to actually consider the location, where it happens.

00:02:55 Is it a dimer or oligomer?

00:02:57 And is it partner-specific?

00:02:58 So it's very sophisticated.

00:03:00 Just think about yourself.

00:03:01 How do you describe your function, right?

00:03:05 Right?

00:03:06 Right?

00:03:07 So in different contexts, when you are interacting with different people, you are fulfilling different functions.

00:03:12 So, and on top of that, we have the incomplete knowledge.

00:03:16 This very famous, very famous quote from, you can tell me who it's coming from.

00:03:25 There's known unknowns and there's unknown unknowns.

00:03:30 Who's this quote coming from?

00:03:33 Well, we have a war starting today, actually.

00:03:40 And this quote was made years ago when another war started.

00:03:48 Okay, you can find it out.

00:03:54 Okay?

00:03:55 So, in short, the conclusion is that you believe that you can collect more and more data and

00:04:04 understand the system more and more so that you would have a very sophisticated, gigantic

00:04:11 model describing the complex system.

00:04:16 And you are just trying to make sure that happens.

00:04:20 Well, it's very difficult.

00:04:22 Okay?

00:04:23 If you learn control, you know that the best control strategy is not necessarily built upon the precise,

00:04:30 I'm sorry, built upon more detailed description of the system.

00:04:36 Okay?

00:04:37 Sometimes it depends on the resolution of the data.

00:04:39 So, based on this conclusion, what machine learning is trying to do is the following.

00:04:51 Machine learning is extremely powerful when you have a system, a complex system, but you have a lack of laws and details.

00:05:04 Right?

00:05:05 As you said, you don't even know all the components involved, and you don't know how they are

00:05:09 interacting with each other in details.

00:05:12 Or sometimes you have a law, you know, the energy minimum principle, but you don't have the details to calculate the Schrodinger equation in a reasonable time.

00:05:27 Right?

00:05:28 In contrast to the lack of laws and details in this natural system, you have abundance of data.

00:05:36 This data is the paired input-output examples.

00:05:41 So you have a lot of IO pairs.

00:05:43 So this is the system that you know machine learning got to be the way because it's too hard to be modeled precisely following principle,

00:05:53 because there's a lack of principle and details.

00:05:56 But in the meantime, you have abundance of data.

00:05:59 So now let's look at this system.

00:06:01 Right?

00:06:02 This is a natural process.

00:06:03 Right?

00:06:04 Whose input includes X and X prime.

00:06:09 And X and X prime, let's think about this process as a causal process.

00:06:14 Okay?

00:06:15 And X and X prime can result in the output.

00:06:19 Let's call it a Y or F of X and X prime.

00:06:23 But you don't know who X prime are.

00:06:26 Or you have no way to measure X prime.

00:06:30 So all you have are a bunch of X and F of X X prime pairs.

00:06:37 Okay?

00:06:38 So that's what you have.

00:06:39 So based on such data, your goal is to construct an artificial model using some learning algorithm and domain knowledge.

00:06:51 Right?

00:06:52 And you want to make a prediction from X.

00:06:55 Notice that it's from X alone to the prediction Y hat or H.

00:07:01 Right?

00:07:02 And usually you want construct such or train such model to replicate the observed IO relationships from the natural processes.

00:07:14 Once you have such model trained, then you can make these predictions with different X.

00:07:20 So this is what machine learning is good at.

00:07:25 Okay?

00:07:26 So many of you have taken or are taking machine learning courses.

00:07:33 So I'm not going to repeat too many repeated knowledge.

00:07:38 But I just want to remind you that there are a lot of things that are important in machine learning, including theories.

00:07:47 Okay?

00:07:48 And we know already from optimization that theories are really important like convergence, convergence rate, complexity.

00:07:57 They actually help us in the way of developing algorithms.

00:08:02 Because without them, it's hard to guide your development.

00:08:04 Is that right?

00:08:05 Because you can only trial and error.

00:08:07 But that price is something very expensive.

00:08:12 So, you have learned bias variance trade-off, right?

00:08:20 Have you?

00:08:22 Bias variance trade-off?

00:08:25 Yes.

00:08:26 Yes.

00:08:27 Okay.

00:08:28 So this is a very important concept.

00:08:30 And we'll discuss them more in details.

00:08:33 But I believe you have learned and or are learning.

00:08:37 Right?

00:08:38 I know more than two or three of you are taking machine learning courses this semester.

00:08:42 Right?

00:08:43 So there is a very important concept about the complexity

00:08:49 of the model and the generalizability of the model.

00:08:53 So that's what the trade-off is trying to address.

00:08:55 And then knowing that the complexity is important

00:08:59 to the generalizability, then we can regularize

00:09:02 or control the complexity to affect the generalizability.

00:09:08 This will be discussed, and at least repeated

00:09:11 in this lecture, in our courses.

00:09:14 Then there are a lot of other things, right,

00:09:16 including the time complexity, including learnability, right?

00:09:21 I believe that PAC learnability is definitely mentioned

00:09:25 or will be mentioned in machine learning with networks.

00:09:27 So probably approximately correct PAC learnability.

00:09:35 So these are the theories involved.

00:09:37 And I have very incomplete list of machine learning algorithms

00:09:42 because here these are just the focus on supervised

00:09:45 versus unsupervised.

00:09:47 But you can also think about the different modalities, right,

00:09:52 involved.

00:09:53 And, you know, here I just put everything under deep learning,

00:09:57 but you know that it's way more than that.

00:09:59 So luckily we have a whole third module, right, almost 50% of the

00:10:04 lectures on the deep learning part, okay?

00:10:08 All right.

00:10:09 Any questions so far as we start on this part?

00:10:15 Okay, let's have a formal setup, okay?

00:10:18 So as I said, what we have is a list of paired X and Y's input and outputs, okay?

00:10:28 So let's say the input data is drawn from a space X and the output, also called label or target,

00:10:37 is drawn from a space Y, okay?

00:10:40 So X is also called the feature space, right?

00:10:43 Sometimes you can roughly say that each dimension is a feature,

00:10:47 but it also depends because sometimes people think that X, each dimension is just the attribute.

00:10:52 So what feature means is something that you extracted from the attributes.

00:10:59 So those are just the names.

00:11:01 So if the space, the target or the label space, output space is finite,

00:11:10 then we're talking about a classification problem, right?

00:11:13 You have finite number of labels, but if it's continuous,

00:11:16 we're talking about a regression problem.

00:11:18 And the unknown function we're going to try to learn is from X to Y.

00:11:24 So it's a mapping between the two spaces.

00:11:27 And as I said earlier on, we are given a set of IO pairs.

00:11:33 So the input is Xi, it's bold-faced, meaning that it's a vector.

00:11:37 And the output is Y sub i, meaning that it's a scalar, it's not bold-faced.

00:11:42 So for now we just consider a scalar variable, sorry, a scalar label.

00:11:48 And we have N such so-called examples, okay?

00:11:54 Paired input-output relationships, okay?

00:11:57 So imagine that you are a hiker, you are a tea drinker, you notice that, right?

00:12:04 The time needed to boil water to make your tea is different when you are at home

00:12:13 or when you are at the top of the mountains, right?

00:12:15 Though this, right, you may not know the details of the physics yet for now, okay?

00:12:20 I'm assuming, but of course in this case, in the physics is not that sophisticated.

00:12:24 So you don't need the machine learning to tell that.

00:12:26 But anyway, let's say this was before the discovery of that physics.

00:12:31 So what you can do is, you know, you can just look at different locations

00:12:36 and measure the pressure at that location, that's the input,

00:12:41 and also measure the boiling temperature of boiling point at that location, that's output.

00:12:49 Then you can find out that there's some, yeah, almost linear.

00:12:53 You don't need the machine learning, but your brain is already doing machine, right?

00:12:57 Human learning.

00:12:58 There's a model you think that, oh, it looks like linear, right?

00:13:02 Believe it or not, our brains are regularizing the complexity of the models with our own priors, okay?

00:13:09 So here we actually think that a linear model probably would suffice, okay?

00:13:16 But I think in machine learning, we are doing things in a more clear way.

00:13:21 We don't know fully how brains function, okay?

00:13:25 Okay, so this is a formal setup.

00:13:27 And again, you have learned all this before.

00:13:31 I don't want to talk too much about it.

00:13:33 We are talking about supervised versus unsupervised.

00:13:37 And then in the case of the supervised, depending on the label,

00:13:40 we have classification versus regression.

00:13:42 And then in unsupervised, actually, I list some examples.

00:13:47 But what is not included included this unsupervised learning and self-supervised learning in deep learning.

00:13:55 So those will be partially covered in the deep learning module, okay?

00:14:01 Okay, so let's see.

00:14:06 Let me ask you a question.

00:14:08 What kind of machine learning problem are we dealing with when we try to derive statistical potentials?

00:14:17 Remember, we are using statistical potentials to replace the physics-based energy models like force fields

00:14:25 so that we can optimize that function to find the best structure prediction.

00:14:30 So what kind of learning algorithm did we use to derive the statistical potentials?

00:14:36 Probability of the sequence, sorry, probability of the structure given the sequence?

00:14:49 Well, what feature do you have for each structure?

00:14:55 Some kind of embedding of the structure, okay?

00:15:00 But what label do we have?

00:15:03 The label is how close this structure is compared to the native structure, which is the ground truth, right?

00:15:11 Let's call it RMSD, root mean square deviation.

00:15:16 Remember, when we compare the structure similarity, we use the root mean square deviation, okay?

00:15:21 So if I use the root mean square deviation as a label and learn a function to reconstruct such relationship,

00:15:28 I am doing a regression problem, right?

00:15:32 But I could also classify this, you know, binarized label based on the root mean square deviation, right?

00:15:39 That's what David Baker did in the earlier Rosetta.

00:15:42 So they look at the root mean square deviation, if it's less something, they say it's near native.

00:15:47 If it's bigger than something, it's, you know, not native.

00:15:52 And then he used the logistical regression for the classification problem, right?

00:15:57 So binary classification.

00:15:58 So it depends, okay?

00:16:00 Now, what kind of learning are we talking about for protein function prediction?

00:16:05 Is it supervised, unsupervised?

00:16:11 Is it classification or regression?

00:16:14 Huh?

00:16:19 Unsupervised.

00:16:20 Supervised.

00:16:21 Okay, why?

00:16:23 Why unsupervised?

00:16:25 There's a huge amount of data that is not labeled.

00:16:42 Okay.

00:16:44 I see what you mean.

00:16:47 Okay, let me explain a little bit.

00:16:49 Sorry for the confusion.

00:16:50 When I put no labels down there below unsupervised learning, what I meant was not that what you want

00:16:58 to predict for does not have labels.

00:17:01 And what I meant was that in the process of making a prediction or learning a model, there's

00:17:11 no label data you utilize.

00:17:13 You see what I mean?

00:17:15 So when you construct that model for machine learning, there's no Y.

00:17:20 There's only X.

00:17:21 I'll give you one example, right?

00:17:24 For instance, I look at, you know, the Google Earth image of the world at night, right?

00:17:37 Then I see a lot of lights, right?

00:17:40 And then the size of, I'm sorry, the size of the light, I'm sorry, the intensity of the light

00:17:45 is like how much human population is there, right?

00:17:50 Activities are there.

00:17:51 Sorry, activities.

00:17:52 Now, not necessarily population because there are populations who don't have a lot of access

00:17:56 to electricity, right?

00:17:57 And then the size is sort of showing the metropolitan area size, right?

00:18:02 So in that case, my X are what?

00:18:06 Locations and intensities of these lights, right?

00:18:10 And I do clustering and I found the big cities, right?

00:18:14 So I didn't use any Y.

00:18:16 You see what I mean?

00:18:18 So in that case, we say it's unsupervised, okay?

00:18:22 So is that clear?

00:18:27 Okay.

00:18:28 So I'm going to come back to you, okay?

00:18:31 So let's ask Doyoung again.

00:18:34 Why do you think it's a supervised learning problem?

00:18:37 Good.

00:18:38 So you're saying that, oh, we do have some percentage of protein sequences whose functions

00:18:53 are already labeled, right?

00:18:55 Okay.

00:18:56 Then is that a classification problem or regression problem?

00:19:00 What are the labels?

00:19:01 Is the label space discrete or continuous?

00:19:14 Discrete.

00:19:15 Discrete, because we are looking at geo terms on the gene ontology, right?

00:19:20 So in that sense, you can say it's discrete or classification.

00:19:24 But you can also, just like a logistic regression, right?

00:19:27 It's for classification, but it calculates the continuous probabilities in each class.

00:19:32 So in this case, you can also do this multi-class classification, right?

00:19:36 Okay.

00:19:37 But now, thank you for the answer.

00:19:40 Now let's come back to what Ryan said.

00:19:43 Actually, it's also correct.

00:19:46 Because there are a lot of data for protein sequences without labels yet.

00:19:52 So does that mean we cannot use them?

00:19:57 Can we still use them?

00:20:02 How do we use them?

00:20:05 Can you cluster proteins based on their gene expression?

00:20:22 Then you assume that genes in the same cluster should have the same or similar function.

00:20:33 Then that's also the approach, right?

00:20:36 So in reality, you could combine these things together, okay?

00:20:39 So it depends on your formulation.

00:20:42 It's not a fixed scenario.

00:20:44 So there's no correct, single correct answer for this question.

00:20:48 I hope this shows you the, you know, this is part of your design.

00:20:53 Okay.

00:20:56 All right.

00:20:57 So as I said, protein function prediction, it can be a supervised classification problem,

00:21:02 where the input are the features and the output are labels, which are the leaf terms of the geo terms.

00:21:08 So it's the most detailed gene ontology.

00:21:12 So it's supervised classification.

00:21:14 It can be supervised regression, right?

00:21:17 The labels is likelihood between 0 and 1 for each leaf.

00:21:22 But it's not really regression.

00:21:24 It's still classification.

00:21:25 Okay.

00:21:26 Okay.

00:21:27 So just, but the, the, the, what do you predict?

00:21:30 It can be different, right?

00:21:32 Um, another one is unsupervised classification.

00:21:36 You can group related genes based on the similarity, right?

00:21:40 And this, you can, this also can be, uh, semi-supervised.

00:21:45 In other words, you can use, uh, unlabeled test set, including, so, so, so, so you can also use unlabeled test set.

00:21:55 In other words, for the test set, you don't know their labels, but you can use their features.

00:22:00 And then you can use the semi-supervised learning, right?

00:22:03 You, you have learned what semi-supervised learning means, right?

00:22:06 Okay.

00:22:07 So, all right, let's, uh, you know, I don't have to say too much about it, but, uh, in the case of the supervised learning, right,

00:22:21 what you, you really do is, uh, you process the data and, uh, split the data into the training and test set.

00:22:28 You select the algorithms based on the kind of problem you deal with.

00:22:32 So, uh, you know, you're doing the model, uh, finding the parameters based on the training set.

00:22:44 And then you were evaluate over the validation set, uh, what they say test here, I think should be validation or development.

00:22:51 And then you find out which hyperparameter is the best and the, you know, the optimal parameters also associated with that, um, best hyperparameters.

00:23:02 And then, uh, eventually you find a best model or ensemble of models and, uh, you can, uh, deploy that.

00:23:11 Okay.

00:23:12 So when you choose an algorithm, you can, based on, uh, evaluating the performances, uh, cross-validation, leave one out.

00:23:21 Uh, if you are taking pattern recognition course, you will learn about more details and theoretical, uh, aspects, especially when the sample size is small,

00:23:33 whether cross-validation or leave one out is still a good idea.

00:23:37 So you will learn a lot, okay, from, uh, Professor Baganato in that course.

00:23:42 And then, uh, you could also do the statistical comparisons, right?

00:23:46 For instance, uh, rock curve for classification.

00:23:49 We're gonna remind you again for that.

00:23:51 Okay.

00:23:52 Okay.

00:23:53 So, uh, how to represent the input.

00:23:57 I said in the, you know, I asked this multiple times, uh, and, uh, we have seen an example in the threading, uh, problems.

00:24:06 And we also asked you a couple of times about, uh, how do you represent the protein structure for deep learning model, right?

00:24:13 So, and that actually determines your, uh, model choice, right?

00:24:18 Do you represent them as, um, point clouds?

00:24:23 Do you, uh, represent them as a rigid body?

00:24:27 So it's just a volume.

00:24:29 Uh, do you represent them as just a surface, right?

00:24:34 Manifold?

00:24:35 Uh, you could have different choices, right?

00:24:37 Indeed, they are being represented in different ways, right?

00:24:40 Or you represent them as graph with residues interacting with each other on the, on the nodes, right?

00:24:46 So we learned this before that, uh, you could represent, um, so I know that we talk about, uh, the fact that we can use end-to-end models, right?

00:24:57 And similarly, you know, feature engineering can be reduced.

00:25:03 But honestly, um, feature engineering is still very important.

00:25:08 Uh, it's important to have the domain meaningful features, okay?

00:25:14 Because if you want your model, naive model, to learn such knowledge directly from abundant data, it's very difficult.

00:25:25 Okay, give you one simple example.

00:25:28 If you don't tell the model in a way that, you know, this, uh, label is, uh, invariant to the translation of the object, X.

00:25:43 For instance, however I translate the image, or the cat in the image, it's still a cat.

00:25:50 Sorry, did I say translate?

00:25:52 Rotate, sorry.

00:25:54 Right?

00:25:55 So, convolutional neural networks can achieve the translational invariance.

00:26:00 That's not a problem, right?

00:26:02 So we'll talk about this in more details.

00:26:04 But it doesn't know rotational invariance, does it?

00:26:07 Right?

00:26:09 So, so how do people usually teach a convolutional neural networks about the rotation invariance?

00:26:17 Usually they just have to let this model learn from a lot of data.

00:26:23 For each X, Y pair, you randomly rotate X many times and so-called augment the data set, training set.

00:26:32 Right?

00:26:33 Then the model hopefully sees that, oh, I have so many examples that their X are different but rotated relative to each other and their Ys are the same.

00:26:41 Okay?

00:26:42 But still, it's a very low efficiency.

00:26:47 Okay?

00:26:48 And there's no guarantee that it's, you know, not to mention that you're making this learning process extremely painful.

00:26:56 Okay?

00:26:57 And so for that sense, you do have to do some feature engineering sometimes.

00:27:04 And that does matter.

00:27:05 Okay?

00:27:06 So we talked about this before.

00:27:07 You can represent them in sequence, profile, and structure.

00:27:10 But even for sequence, right?

00:27:12 So there are a lot of things than just doing a one hot.

00:27:16 Right?

00:27:17 In homework three, you will learn something about it.

00:27:20 You will see how you choose your input representation really matters.

00:27:26 And if you just tell them that, oh, these are just the 20 different letters, then it's extremely difficult for the machine learning model to figure out how to assign a sequence to a label.

00:27:40 Because it has no idea that this letter actually corresponds to a hydrophobic small residue.

00:27:47 And that letter is also a hydrophobic but a bulkier residue.

00:27:53 It has no concept at all.

00:27:56 Imagine that you just, the first time you learn the alphabet, if you just look at these 26 letters, right, what can you say?

00:28:06 The only thing you can say is they are different.

00:28:08 Okay?

00:28:09 You have no knowledge.

00:28:11 And how many examples after that did you need to finally have a concept, right?

00:28:18 But in that case, we still have rules, right?

00:28:21 So it's important to represent the inputs and incorporate the domain knowledge.

00:28:28 So the domain knowledge is in your input.

00:28:30 I'm sorry.

00:28:31 It's first of all in your design of the, or formulation of the problem.

00:28:36 Then it's in your input.

00:28:38 Then it's also in your model architecture accordingly.

00:28:42 So that's where you can introduce the domain knowledge.

00:28:48 And lastly, what I didn't say was the last function, the objective function, right?

00:28:52 That's one of the most important things for the optimization problems.

00:28:56 You could represent inputs differently than just using features.

00:29:06 Okay?

00:29:07 Instead of just representing each x with a lot of features, you can actually also use something called kernels.

00:29:16 Okay?

00:29:17 Kernels is, you know, you can roughly think of it as a distance.

00:29:23 Okay?

00:29:24 So in other words, I don't tell you what is the coordinate of each sample x in the space.

00:29:33 But I do tell you what's the distance or similarity or dissimilarity between two, between a pair of examples, xi and xj.

00:29:44 You don't know where, what are the coordinates or the features of xi and xj.

00:29:49 But you do know the distance or similarity or dissimilarity between xi and xj.

00:29:54 Is that clear?

00:29:55 So that's what the kernel representation means.

00:29:58 Okay?

00:29:59 And this could be a Euclidean distance in the protein feature space, but this can be a bad choice.

00:30:07 Right?

00:30:08 Because you stack all the different features into a space and then different dimensions can be different types of variables and the scales are different.

00:30:20 If you just use the Euclidean distance in that space, it doesn't make any sense.

00:30:25 Right?

00:30:26 But in your applications, very often, people have some domain knowledge defined distance concept.

00:30:35 Right?

00:30:36 Like in our structure similarity, we know how to measure structure similarity.

00:30:40 Right?

00:30:41 So in other kind of analysis, you have some idea about what similarity means.

00:30:46 Right?

00:30:47 So you want to teach that to the machine learning model.

00:30:50 Right?

00:30:51 Then this, you know, kernel representation is another way.

00:30:55 We'll actually see the kernel representation in the support vector machines again.

00:31:00 Okay?

00:31:01 So, and I want you to carry that powerful concept when we review the kernel methods.

00:31:09 And how we carry that on all the way to deep learning, that's another very active line of research.

00:31:16 Okay.

00:31:17 So, yes, as I said, these kernel methods are very often used.

00:31:23 Like, k nearest neighbors.

00:31:25 Right?

00:31:26 You don't care about the features.

00:31:27 All you care about is who are my neighbors.

00:31:29 Right?

00:31:30 And then the support vector machine as well, actually.

00:31:33 It's a kernel method.

00:31:34 Okay?

00:31:35 All right.

00:31:37 So, now let's finish this motivation part about why we want to do machine learning.

00:31:43 Right?

00:31:44 Generally, if there's a lack of general laws or specific details for a very complex system,

00:31:52 but for that system you have abundance of I.O. data, then it's a perfect scenario for machine learning.

00:32:02 Because you can construct an artificial model to reconstruct the I.O. relationship and try to make generalization.

00:32:10 So, that's when you use machine learning.

00:32:13 Okay?

00:32:14 But you don't use machine learning to discover a principle or, you know, F equal to M.A. from data.

00:32:22 Because you already know that, you just want to incorporate that directly into your model.

00:32:26 Okay?

00:32:27 And then we talked about a feature, label, training set, test set, different kind of learning scenarios,

00:32:33 and what theories and algorithms might be involved, mostly in the shallow, because the deep learning part will be in different modules.

00:32:40 [Uncertain recognition] Right?