Transcript
Use with ChatGPT / Claude Auto-scroll A
Introduces the linear algebra for AI course, its goals, and the sequence of topics covered. Also explains the module structure and the role of exercises.
Hello everyone, and welcome to linear algebra for AI, part of the foundations for machine learning program at KAUST Academy.My name is Haben Al-Khori, and I'm going to be your instructor for this course.In this course, we will explore some of the fundamental ideas of linear algebra and see how they are used in AI and machine learning.So, the goal is not just to learn mathematical formulas, but to understand how these mathematical tools help us represent data, make prediction, and solve problems.So, we will focus on the concepts that are most relevant to machine learning using simple examples and practical applications along the way.
So, let's talk about the route.Here's the route for our course.We begin with vectors and their geometry, then we move on to uh matrices, and matrices will allow us to organize and process large amounts of data.We then look at system of linear equations, followed by column spaces, rank, and projections.Finally, we will connect all of these ideas to the least squares solutions and the linear regression.So, we'll see how linear algebra can be used to make predictions from data.Each module will that will be also followed by six exercises that I highly recommend for you guys to do.So, we going to begin with module number one.Module number one is vectors and geometry.In vectors and geometry, we are going to compute similarities, distances, and make simple predictions.Then, our end goal will be to make more predictions by using the least squares solution.
Defines vectors, their dimensions, and common notation used throughout the course. It also covers zero vectors, one vectors, and standard unit vectors.
So, let's begin with vectors.What are vectors?A vector is an ordered list of number and it's written as a column vector or a row vector.So, a column vector as you see on the left-hand side, you see a vector that represents four entries and it's written between square brackets or between parentheses, the middle one.And then finally, the vector that is in a row vector and we have only four entries in each one of them.So, they can be represented in different ways.The numbers in the list are the entries or components.The number of elements in every list will determine the size of the dimension of the vector.In this case, the size or the dimension of the vector is equal to four.Example.Here's an example of a house as a data point.So, since you are going to learn machine learning, you are going to represent your data points in vectors.
So, we describe each house using two features.So, X1 is the area, that's the feature one.It represent the area of the house in square meters and X2 is the number of bedrooms where they can be between 1 2 3 4 5 and so on.So, the example of X which is equal to 120 and 3 represent a house with area 120 square meters and with three bedrooms.So, this house has three bedrooms.So, every house will be represented as one vector in R2.What we mean by R2?This is the two-dimensional space.It has a X value and a Y value, but we represent the data also as X, so don't get confused.So, uh X is represented as 120 and 3 and it has two components in the direction of I, as you used to know, and in the direction of J.Vector notations, so we often use symbols to denote vectors, and these symbols are represented as X, A, U, P, beta, and capital X.
But, if you noticed, we're using the bold.So, bold X and bold A.So, when you have a bold letter, then this will represent a vector.When bold U is used, we denote by UI its element.So, if we have three components, we U will be represented as U1, U2, and U3.For the vector of size N, indices run from 1 to N.In computer languages, indices run from 0 to N minus 1, which you are going to see later on, or you may have seen in your Python course.Sometimes, bold UI refers to the ith vector of a list of vectors.We are going to see that in matrices later on.So, this bold UI is no longer a component because it's it's represented in a bold letter U with the index I, it refers to the ith vector.So, you may have K vectors, you denote them by bold U1 all the way to bold UK.
We have some special vectors that appear very often, which are the zeros, the ones, and the unit vectors.So, a an N vector with all entries zero is denoted by zero sub N or just zero.People write down zero vector.How is the zero vector represented as just zero, but it follows the dimension of all the vectors you are working with with entries just zero.The one vector has entries one.So, if I'm working with vectors of dimension or size n, I will have one all the way n times.A standard unit vector, we have the standard unit vector which are the e i's.The e i's are the vectors where zeros are everywhere except at the i'th position.So, at the i'th position.As a simple example, which I'm going to write here, e1 is 1 0 0 in R3.e2 is 0 1 0 in R3.
e3 is 0 0 1.Okay?So, these vectors are uh standard unit vectors.And we will understand a little bit what unit means in a moment.And they represent previously the i and j and k axis.
Explains vector addition, scaling, subtraction, and the idea of linear combinations. The speaker shows how combining scaled vectors produces new vectors geometrically.
Next, we look at how we can add vectors.For vectors of the same size, we can add u + v.So, u + v is going to be a third vector of the same dimension of u and v or the same size.So, dimension or size is the same thing.So, u + v is a summation of those two vectors where you're adding every single corresponding component of u with a corresponding component of v.So, u + v is u1 + v1, u2 + v2, and so on.In R2 or R3, you can always represent it geometrically.So, if you have the figure on in the middle of your slide, you have u is one vector in one direction, v is another vector, then the summation would be the diagonal of the parallelogram formed by U andV.Vector addition and vector subtraction, we are going to see how we can subtract vectors.
In order to do that, we need scaling.The scaling the scaling of a vector is given by multiplying this vector by a constant.This constant can be zero, can be negative or positive, can be an integer, a real number, and so on.So, CU is equal to CU1, CU2 all the way to CUN.So, scaling a vector meaning multiplying every single component by the scalar C.If C is greater than 1, then your new vector is longer.So, if we're looking at the figure, you have U that is coming from the origin O.And when C is equal to 2, 2 multiplied by U is a longer vector.You are doubling your vector U.
C is between 0 and 1, so it's a shorter it's a shorter vector.So, suppose let's take an example, C is equal to 1/2 of U, so you are cutting U into half.C is negative.When C is negative, it means that U is in the opposite direction.For example, take C is equal to -1, so I have a vector in the opposite direction of U, and I would have the negative U as you can see in your figure that is in the same magnitude ofU, but it is going in the opposite direction.So, now we can subtract two vectors.What does it mean subtracting two vectors?It means scaling one of the vector by negative a negative number, and then doing U minus V.So, we scale first and then we do an addition.Next.Let's see in more details later on in this slide where we talk about linear combinations.
What are linear combination?So, a linear combination is an expression that is obtained by scaling vectors and then adding them.So, in this case, what did I do?We scaled you, we scaled v.So, 2u + 3v.So, if you're looking at your figure, you have u and then you want to double u, so you are going to create a vector of magnitude that is twice as u.The same thing happens with v.In the direction of v, you're creating three vectors of v.And then you're going to add them at the end.So, when you add them, what are you going to get?You're going to get the parallelogram, the diagonal of the parallelogram, and that would be your answer.So, if we go back to the subtraction that I was talking about in in a moment.So, when you're doing u minus v for instance.
So, if I want to do u minus v, so what do I do?I have to scale v by -1.So, if I have if v is in this direction I'm going to have to scale v by -v, and then I'm going to have to add u into it.So, adding u into it, you are that you're going to get you are going to create your parallelogram and you will have u minus v.So, this is -v as well is just the diagonal of the parallelogram.Of course, my figure is not to scale, but the idea is just exactly the same as the linear combination that we have seen in this particular example.In general, as we see, in general, so we're looking at the set of all linear combinations of certain vectors ui is given as c1u1 all the way to cmum.You don't need to only add two vectors, but you can add more than two vectors.
You can add m vectors.So, this is a linear combination where the c i's are just called the coefficients.
Describes what linear combinations can generate, including lines, planes, and spans through the origin. The section ends with an RGB color example illustrating linear combinations in practice.
What can linear combination produce?So, for a non-zero vector, the set of all linear combinations of the vector u, which are denoted by cu, where all the c are in R, it gives a line through the origin.So, if you start by drawing the vector u, and you take the set of all linear combination of u, let's take some examples.For example, 2 * u will be on the same line that is supported by u.3 * u will be on the same direction of u and on the same line that is supported by u.The same for negative u, -2u, and so on.And don't forget your scalar is a real number, so it can be an integer, it can be a positive or negative integer, it can be zero, that's why your line has to pass through zero, it can be arational number, irrational, and so on.
So, you will obtain the line passing through the origin.So, the set of all linear combination of the vector u is a line through the origin.How about we took two vectors, two vectors that are non-zero vectors?So, for two vectors u and v, the set of all linear combination cu + dv will give you a plane.So, suppose you have the vector u in the following way, we are in R3, and then the vector v.So, all this linear combination is going to give you the diagonal of the parallelogram formed by combining u and v, cu + dv, and the whole answer will give you a plane.But, in condition, where does this plane pass through?It has to pass through the origin zero.Why it has to pass through the origin zero?Because if c is zero and D is zero, then we'll get zero times U plus zero times V, the zero vector, is a linear combination of U and V.
So, the set of all linear combination of what one vector is a plane through the origin, the set of linear combinations of two vectors is a plane that pass through the origin.And this set of all linear combination is called the span of vectors.So, this is called the span of vectors.You are going to see this terminology coming over and over again whenever we're moving more into the material.So, the span of U and V is just the set of all linear combination as we we uh denoted them, CU plus DV.So, it's the set of all linear combination of U and V.A good example of linear combinations are the colors on the RGB color model.And this can be created by layering all three primary colors on top of each other at different brightness.So, RGB stands for red, green, and blue.
So, the red color is represented by a vector of three dimensions uh with components 1 0 0, and green with components 0 1 0, and blue with component 0 0 1.If we mix red and green, we are going to get yellow.What do you think mixing red and green means?It means adding one times the vector red plus one times the vector green, which you is going to give you 1 1 0, and that would be the yellow color.Mixing RGB to get gray, how can we get gray?Well, how do you think we can get this gray color?We multiply 0.5 by the red color plus 0.5 by the vector representing the green color and 0.5 by the vector representing the blue color.To obtain 0.5, 0.5, and 0.5 as components that will represent the gray color.