Chapter 5Principal Components Analysis, Residual Dipolar Couplings, and Their Relationship in NMR Structural Biology

Anthony K. Yan and Bruce R. Donald

In biochemistry, nuclear magnetic resonance (NMR) is a powerful tool for analyzing the structure and dynamics of proteins in solution. Recently, research has taken advantage of a type of NMR data known as residual dipolar couplings (RDC). The physics of RDC’s tells us they are formed from a time and ensemble average. This ensemble average is mathematically similar to the data analysis technique known as principal components analysis (PCA). In this chapter, we consider the formal relationship between the two, and consider the information content of the RDC formalism with respect to protein flexibility. After mastering this chapter, you will understand the fundamentals of RDCs, which are important in structural biology, and know the foundations of PCA, one of the most important algorithms in computational science and applied statistics.

In classical physics (or molecular dynamics, MD), ensemble averages are over continuous degrees of freedom. To keep the discussion simple for our broad audience, in this chapter we pretend that the ensembles are discrete. By way of intuition, in quantum mechanics the ensemble is discretized, so our derivation is analogous to averaging over quantum states.

At any rate, the reader who is familiar with continuous probability distributions and Jacobians will find it easy to generalize all the math in this chapter to continuous ensembles. Basically, the key ideas in the lecture are very simple, and we didn’t want to have intimidating integrals everywhere: This is why we feel that discrete averages make for a cleaner presentation.

Introduction

Residual dipolar couplings (RDCs) are a type of data from NMR that has been useful for solving protein structures and solving the assignment problem. According to the physics of RDCs, they are based on the relationship between internuclear vectors and a set of parameters known as the Saupe alignment tensor.

In this chapter, we consider the relationship between the Saupe alignment tensor and a data analysis technique known as principal components analysis (PCA). It turns out that the Saupe matrix is mathematically similar to the covariance matrix in PCA. The Saupe matrix is the result of an ensemble average of internuclear vectors, and contains statistical information about the ensemble that is very similar to the statistical information contained in PCA.

Intuitively, PCA gives us the spread or variance of a set of points along the principal axes of the ellipsoid that best fits the point set. One might have hoped that the Saupe matrix would contain exactly the same information as PCA. However, this is not the case. Instead, the Saupe matrix contains an upper bound of the variances of its ensemble. It will be shown that the Saupe matrix along with the ensemble average contains information precisely equal to PCA. However, the ensemble average is not contained within the RDC data; consequently, we are able to compute only an upper bound on the ensemble’s variances. If the average of the ensemble were estimated by alternative means, we could use the Saupe matrix to perform PCA on the ensemble.

Below, we derive PCA, and explain its properties. Although this may be a review to some readers, it is useful for the exposition here as we will also derive the Saupe matrix formalism and explain its relationship to the derivation of PCA.

The structure of this chapter is as follows. First we derive PCA in section 2. In section 3, we derive the Saupe alignment tensor. In section 4, we compare the Saupe matrix to PCA, and explain their relationship. Finally, in section 5, we discuss future work and possible applications. In the appendix, we explain how to efficiently perform PCA on data sets with very large dimension and a relatively small number of data points.

Introduction to PCA

We motivate PCA by considering the statistics of discrete sets of ℝ\mathbb{R} . Let A={ai}A = \{a_i\} , where ai∈ℝa_i \in \mathbb{R} and i∈{1,2,3,…,N}i \in \{1, 2, 3, \dots, N\} . The standard statistical characterizations of AA are the mean and variance (standard deviation):

⟨ai⟩=1N∑i=1Nai(5.1)\langle a_i \rangle = \frac{1}{N} \sum_{i=1}^N a_i \quad (5.1)

σ2(A)=1N∑i=1N(ai−⟨ai⟩)2.(5.2)\sigma^2(A) = \frac{1}{N} \sum_{i=1}^N (a_i - \langle a_i \rangle)^2. \quad (5.2)

Very often, our data is not in ℝ\mathbb{R} , but instead is in ℝm\mathbb{R}^m . We would like to define the direct analogs of mean and standard deviation in higher dimensions. Consider a point set in ℝm\mathbb{R}^m . Let P={𝒑i}P = \{\mathbf{p}_i\} , where 𝒑i∈ℝm\mathbf{p}_i \in \mathbb{R}^m and i∈{1,2,3,…,N}i \in \{1, 2, 3, \dots, N\} . The mean is easy to define:

⟨𝒑i⟩=1N∑i=1N𝒑i.(5.3)\langle \mathbf{p}_i \rangle = \frac{1}{N} \sum_{i=1}^N \mathbf{p}_i. \quad (5.3)

The analog of the standard deviation is less easy to define in ℝm\mathbb{R}^m . However, we can simplify things by projecting our data points PP onto a one-dimensional line. We will then compute the directional variance along the line.

To simplify our presentation, let us assume that our data PP has already been zero-meaned:

⟨𝒑i⟩=1N∑i=1N𝒑i=0.(5.4)\langle \mathbf{p}_i \rangle = \frac{1}{N} \sum_{i=1}^N \mathbf{p}_i = 0. \quad (5.4)

We project PP onto a line through the origin, which is specified by the unit vector 𝒖\mathbf{u} . Let 𝒒i(𝒖)\mathbf{q}_i(\mathbf{u}) be the projection of 𝒑i\mathbf{p}_i onto the line specified by 𝒖\mathbf{u} :

𝒒i=(𝒖⋅𝒑i)𝒖=li𝒖(5.5)\mathbf{q}_i = (\mathbf{u} \cdot \mathbf{p}_i) \mathbf{u} = l_i \mathbf{u} \quad (5.5) where li=(𝒖⋅𝒑i)l_i = (\mathbf{u} \cdot \mathbf{p}_i) is the (signed) length of 𝒒i\mathbf{q}_i along the line specified by 𝒖\mathbf{u} . We can now compute the average and variance of the lil_i . The mean ⟨li⟩\langle l_i \rangle is zero:

⟨li⟩=1N∑ili=1N∑i(𝒖⋅𝒑i)=𝒖⋅(1N∑i𝒑i)=𝒖⋅⟨𝒑i⟩=0(5.6)\langle l_i \rangle = \frac{1}{N} \sum_i l_i = \frac{1}{N} \sum_i (\mathbf{u} \cdot \mathbf{p}_i) = \mathbf{u} \cdot \left( \frac{1}{N} \sum_i \mathbf{p}_i \right) = \mathbf{u} \cdot \langle \mathbf{p}_i \rangle = 0 \quad (5.6) since we assume the 𝒑i\mathbf{p}_i are zero-meaned.

Now let’s compute the directional variance, which we define as follows:

Definition 5.1 If ⟨𝒑i⟩=0\langle \mathbf{p}_i \rangle = 0 , then the directional variance along unit vector 𝒖\mathbf{u} is

σ𝒖2(P)≡(1N∑ili2).(5.7)\sigma_{\mathbf{u}}^2(P) \equiv \left( \frac{1}{N} \sum_i l_i^2 \right). \quad (5.7)

Playing with algebra, we have:

σ𝒖2(P)=1N∑ili2=1N∑i(𝒖⋅𝒑i)2=1N∑i(𝒖⋅𝒑i)(𝒑i⋅𝒖)(5.8)\sigma_{\mathbf{u}}^2(P) = \frac{1}{N} \sum_i l_i^2 = \frac{1}{N} \sum_i (\mathbf{u} \cdot \mathbf{p}_i)^2 = \frac{1}{N} \sum_i (\mathbf{u} \cdot \mathbf{p}_i)(\mathbf{p}_i \cdot \mathbf{u}) \quad (5.8)

=1N∑i(𝒖T𝒑i)(𝒑iT𝒖)=1N∑i𝒖T(𝒑i𝒑iT)𝒖=𝒖T(1N∑i𝒑i𝒑iT)𝒖.(5.9)= \frac{1}{N} \sum_i (\mathbf{u}^T \mathbf{p}_i)(\mathbf{p}_i^T \mathbf{u}) = \frac{1}{N} \sum_i \mathbf{u}^T (\mathbf{p}_i \mathbf{p}_i^T) \mathbf{u} = \mathbf{u}^T \left( \frac{1}{N} \sum_i \mathbf{p}_i \mathbf{p}_i^T \right) \mathbf{u}. \quad (5.9)

(5.10)(5.10)

So we have,

σ𝒖2(P)=𝒖T(MP)𝒖(5.11)\sigma_{\mathbf{u}}^2(P) = \mathbf{u}^T (M_P) \mathbf{u} \quad (5.11)

MP≡1N∑i𝒑i𝒑iT.(5.12)M_P \equiv \frac{1}{N} \sum_i \mathbf{p}_i \mathbf{p}_i^T. \quad (5.12)

The term 𝒑i𝒑iT\mathbf{p}_i \mathbf{p}_i^T is the outer product of 𝒑i\mathbf{p}_i with itself and is an M×MM \times M matrix. It is not to be confused with 𝒑iT𝒑i\mathbf{p}_i^T \mathbf{p}_i , which is the inner product of 𝒑i\mathbf{p}_i with itself, which is a scalar. If PP is zero-meaned, then MPM_P is known as the covariance matrix of PP . For this chapter, we shall say that if PP is arbitrary (not zero-meaned), then MPM_P is the pseudo covariance matrix of PP . The pseudo covariance matrix equals the covariance matrix when PP is zero-meaned. Later on, we shall be forced to consider the relationship between pseudo covariance matrices and true covariance matrices.

Let’s pause now to consider where we are. Given PP , we can compute the covariance matrix MPM_P . Equation (5.11) tells us that given the covariance matrix, we know the directional variance for all directions 𝒖\mathbf{u} . Intuitively, MPM_P is a complete description of the variance of PP in ℝm\mathbb{R}^m .

Now that we have a notion of standard deviation in ℝm\mathbb{R}^m , we should study its geometric properties. First notice that MPM_P is symmetric by construction. This means MPM_P has real eigenvalues and orthogonal eigenvectors. Furthermore, we know that MPM_P is positive semidefinite because 𝒖T(MP)𝒖=σ𝒖2(P)≥0\mathbf{u}^T (M_P) \mathbf{u} = \sigma_{\mathbf{u}}^2(P) \geq 0 for all unit vectors 𝒖\mathbf{u} . This is because each term li2l_i^2 of the sum in equation (5.8) is non-negative. Consequently, MPM_P has non-negative eigenvalues.

Let 𝒗i\mathbf{v}_i be eigenvectors of MPM_P with eigenvalues λi\lambda_i :

(MP)𝒗i=λi𝒗i(5.13)(M_P) \mathbf{v}_i = \lambda_i \mathbf{v}_i \quad (5.13)

𝒗i⋅𝒗i=δij(5.14)\mathbf{v}_i \cdot \mathbf{v}_i = \delta_{ij} \quad (5.14)

λi≥0(5.15)\lambda_i \geq 0 \quad (5.15) where δij\delta_{ij} is the Kronecker delta. In short, MPM_P is our favorite kind of matrix (orthogonal eigenvectors, non-negative eigenvalues).

We now consider the geometry associated with MPM_P . We’ll consider two aspects of MPM_P . First, we’ll make a plot of the directional standard deviation and second, we’ll consider the shape of the associated Gaussian distribution in ℝm\mathbb{R}^m .

Definition 5.2 If ⟨𝒑i⟩=0\langle \mathbf{p}_i \rangle = 0 , then the directional standard deviation along unit vector 𝒖\mathbf{u} is just the square root of the directional variance:

li=𝒖⋅𝒑i(5.16)l_i = \mathbf{u} \cdot \mathbf{p}_i \quad (5.16)

σ𝒖(P)≡σ𝒖2(P)=1N∑ili2.(5.17)\sigma_{\mathbf{u}}(P) \equiv \sqrt{\sigma_{\mathbf{u}}^2(P)} = \sqrt{\frac{1}{N} \sum_i l_i^2}. \quad (5.17)

One can imagine plotting the directional standard deviation for all directions. One would do this by plotting all the points (σ𝒖(P))𝒖(\sigma_{\mathbf{u}}(P)) \mathbf{u} for all unit vectors 𝒖\mathbf{u} . Intuitively, this is a polar plot where, instead of using angles to specify a direction, one uses a unit vector. The radius of the plotted point is the directional standard deviation. For an illustration of this, see figure 5.1, which is a plot in ℝ2\mathbb{R}^2 . If the eigenvalues of the covariance matrix are all identically equal, then the directional standard deviation is isotropic, hence our plot is circular. On the other hand, if the eigenvalues are not equal to each other, then the plot has a lobes and a dimpled structure. Perhaps surprisingly, the plot is not an ellipsoid, but might more aptly be called a “dimpleoid.”

Next, we consider the Gaussian distribution, which is associated with the covariance matrix. We would like to know the shape of a Gaussian distribution which has the same directional standard deviations as our covariance matrix. Let MPM_P be our covariance matrix. Let 𝒩(0,MP)\mathcal{N}(0, M_P) be a zero-meaned Gaussian with covariance matrix MPM_P . It can be shown that the directional standard deviations of 𝒩(0,MP)\mathcal{N}(0, M_P) are equal to the directional standard deviations of MPM_P . The directional standard deviation of a probability distribution 𝒫\mathcal{P} in ℝm\mathbb{R}^m is computed from its covariance matrix, which is the expected value (average) of the outer product:

Figure 5.1 Examples of directional standard deviations in ℝ2\mathbb{R}^2 . In these three examples, we make polar plots of the directional standard deviation for three different covariance matrices. All three plots are in the Principal Order Frame (POF) of the covariance matrix, so that the eigenvectors are along the xx and yy axes. In (a) the eigenvalues are equal. In (b) the eigenvalues are not equal and nonzero, with the yy eigenvalue being smaller than the xx eigenvalue. In (c) the yy eigenvalue is zero. Credit: Bruce Donald and Anthony Yan.

MP=∫ℝm𝒙𝒙T𝒫(𝒙)d𝒙(5.18)M_P = \int_{\mathbb{R}^m} \mathbf{x} \mathbf{x}^T \mathcal{P}(\mathbf{x}) d\mathbf{x} \quad (5.18)

σ𝒖2(𝒫)=𝒖T(MP)𝒖,(5.19)\sigma_{\mathbf{u}}^2(\mathcal{P}) = \mathbf{u}^T (M_P) \mathbf{u}, \quad (5.19) where the integral is over ℝm\mathbb{R}^m , d𝒙d\mathbf{x} is a volume element of ℝm\mathbb{R}^m , and 𝒫(𝒙)\mathcal{P}(\mathbf{x}) is a probability distribution over ℝm\mathbb{R}^m . We have abused notation in equation (5.19) since 𝒫\mathcal{P} is a probability distribution, not a set of points.

To make things simple, let’s work in the POF of the covariance matrix. In the POF, MPM_P is diagonal, and so is its inverse MP−1M_P^{-1} . It is now easy to write down 𝒩(0,MP)\mathcal{N}(0, M_P) explicitly:

MP=[λ1λ2λ3⋱λm](5.20)M_P = \begin{bmatrix} \lambda_1 & & & \\ & \lambda_2 & & \\ & & \lambda_3 & \\ & & & \ddots \\ & & & & \lambda_m \end{bmatrix} \quad (5.20)

MP−1=[λ1−1λ2−1λ3−1⋱λm−1](5.21)M_P^{-1} = \begin{bmatrix} \lambda_1^{-1} & & & \\ & \lambda_2^{-1} & & \\ & & \lambda_3^{-1} & \\ & & & \ddots \\ & & & & \lambda_m^{-1} \end{bmatrix} \quad (5.21)

𝒩(0,MP)|𝒙=Aexp⁡(𝒙TMP−1𝒙).(5.22)\mathcal{N}(0, M_P)|_{\mathbf{x}} = A \exp(\mathbf{x}^T M_P^{-1} \mathbf{x}). \quad (5.22)

The probability distribution 𝒩(0,MP)\mathcal{N}(0, M_P) can be visualized by its level sets. In particular, let’s consider the level set that occurs when the exponent is equal to unity. In this case, we have the set of all 𝒙∈ℝm\mathbf{x} \in \mathbb{R}^m that satisfies 𝒙TMP−1𝒙=1\mathbf{x}^T M_P^{-1} \mathbf{x} = 1 . If we write this explicitly in the POF, we get

𝒙TMP−1𝒙=∑i(xi2λi)=1,(5.23)\mathbf{x}^T M_P^{-1} \mathbf{x} = \sum_i \left( \frac{x_i^2}{\lambda_i} \right) = 1, \quad (5.23) where xix_i is the ii th component of 𝒙\mathbf{x} in the POF; λi\lambda_i is the eigenvalue associated with the eigenvector pointing along the ii th coordinate axis (and is associated with the component xix_i). Recall that in the POF, the eigenvectors are along the coordinate axes.

The level set represented by equation (5.23) is easily recognized as the equation for a hyper-ellipsoid with semiaxes λi\sqrt{\lambda_i} . The length of the semiaxes are equal to the directional standard deviation along the semiaxes, and are also the standard deviations of the Gaussian distribution 𝒩(0,MP)\mathcal{N}(0, M_P) (see figure 5.2).

However, for directions not along an eigenvector, the ellipsoid’s radius is not equal to the directional standard deviation. To illustrate this point, see figure 5.3, which overlays the level set of the Gaussian with its directional standard deviation.

Figure 5.2 Examples of Gaussian ellipsoids in ℝ2\mathbb{R}^2 . In these three examples, we make polar plots of the level sets for three Gaussians associated with three different covariance matrices. All three plots are in the Principal Order Frame (POF) of the covariance matrix, so that the eigenvectors are along the xx and yy axes. In (a) the eigenvalues are equal. In (b) the eigenvalues are not equal and nonzero, with the yy eigenvalue being smaller than the xx eigenvalue. In (c) the yy eigenvalue is zero. Credit: Bruce Donald and Anthony Yan.

Figure 5.3 Overlay of directional standard deviation and Gaussian ellipsoid (level set of the Gaussian) in ℝ2\mathbb{R}^2 . In this example, we overlay a polar plot of the directional standard deviation and a plot of the associated Gaussian ellipsoid (level set). Our plot is in the Principal Order Frame (POF) so that the eigenvectors of the covariance matrix are along the coordinate axes. Notice that along the eigenvectors, the semiaxes of the ellipse are equal to the directional standard deviation. However in directions not along any eigenvectors, the radius of the ellipse is not equal to the directional standard deviation. Credit: Bruce Donald and Anthony Yan.

Finally, we discuss the largest and smallest directional variances. Let 𝒖\mathbf{u} be a unit vector in ℝm\mathbb{R}^m , and let PP be an arbitrary point set:

σ𝒖2(P)=𝒖T(MP)𝒖=λ1u12+λ2u22+λ3u32+…+λmum2(5.24)\sigma_{\mathbf{u}}^2(P) = \mathbf{u}^T (M_P) \mathbf{u} = \lambda_1 u_1^2 + \lambda_2 u_2^2 + \lambda_3 u_3^2 + \dots + \lambda_m u_m^2 \quad (5.24)

𝒖T𝒖=u12+u22+u32+…+um2=1(5.25)\mathbf{u}^T \mathbf{u} = u_1^2 + u_2^2 + u_3^2 + \dots + u_m^2 = 1 \quad (5.25)

ui2≥0.(5.26)u_i^2 \geq 0. \quad (5.26)

From equations (5.24)–(5.26) we see that in general, σ𝒖2(P)\sigma_{\mathbf{u}}^2(P) is a convex sum of the λi\lambda_i ’s with coefficients ui2u_i^2 . That is, σ𝒖2\sigma_{\mathbf{u}}^2 is a weighted average of the λi\lambda_i ’s. As is well known, the convex sum (convex combination) of mm points lies within the convex hull of the mm points. Here, we apply this fact trivially, because our points λi\lambda_i live in only one dimension, ℝ\mathbb{R} . So our “convex hull” is simply the largest and smallest values of λi\lambda_i :

λmin≡miniλi(5.27)\lambda_{\min} \equiv \min_i \lambda_i \quad (5.27)

λmax≡maxiλi(5.28)\lambda_{\max} \equiv \max_i \lambda_i \quad (5.28)

λmin≤σ𝒖2(P)≤λmax.(5.29)\lambda_{\min} \leq \sigma_{\mathbf{u}}^2(P) \leq \lambda_{\max}. \quad (5.29)

PCA is the procedure of computing the eigenvalues λi\lambda_i and the corresponding eigenvectors 𝒗i\mathbf{v}_i , and is used to characterize the directional variances of the data PP . It is beyond the scope of this book to give a full exposition of PCA. For additional details and applications, we refer the reader to any number of books on statistics, dimensionality reduction, and clustering. It is hoped, though, that this brief introduction will give the reader a solid intuition for PCA. (See the appendix for how to efficiently compute PCA in high dimensions with relatively few data points.)

Residual Dipolar Couplings in Structural Biochemistry

In NMR, we have the interaction of nuclear spins. Intuitively, each spin can be thought of as a magnetic dipole (a tiny magnet) that is spinning like a gyroscope. If there is a very strong external magnetic field, then the dipoles will align themselves with the external field.

In this case, the dipole-dipole interaction of spins can be written as

D=μ0γaγbℏ4π2rab3⟨3cos⁡2θ−12⟩(5.30)D = \frac{\mu_0 \gamma_a \gamma_b \hbar}{4\pi^2 r_{ab}^3} \left\langle \frac{3 \cos^2 \theta - 1}{2} \right\rangle \quad (5.30)

D=K⟨3cos⁡2θ−12⟩(5.31)D = K \left\langle \frac{3 \cos^2 \theta - 1}{2} \right\rangle \quad (5.31) where γa\gamma_a and γb\gamma_b are gyromagnetic ratios of the two spins, ℏ\hbar is Planck’s constant, rabr_{ab} is the distance between the two spins, and θ\theta is the angle between the external field 𝑩\mathbf{B} and the internuclear vector 𝒗\mathbf{v} (figure 5.4). KK is a fixed constant equal to μ0γaγbℏ/4π2rab3\mu_0 \gamma_a \gamma_b \hbar / 4\pi^2 r_{ab}^3 . The angle brackets represent an average over time and over the ensemble of proteins in solution, where the scale of the time average is much shorter than can be measured experimentally. For simplicity, we assume here that the nuclei aa and bb are covalently bonded, so the distance rabr_{ab} is not an ensemble average but known from crystal structures.

We now play a series of algebra games, which are the standard manipulations for deriving the Saupe matrix formalism. This is basically the same algebra we used to derive the covariance matrix, where a dot product (𝑩⋅𝒗)2(\mathbf{B} \cdot \mathbf{v})^2 or a cos⁡2θ\cos^2 \theta term is expanded as an outer product. To simplify the derivation, let’s normalize 𝑩\mathbf{B} and 𝒗\mathbf{v} :

Figure 5.4 Dipole-dipole interaction in a strong external field. If an external magnetic field 𝑩\mathbf{B} is very strong, it will align the two spins SaS_a and SbS_b ; 𝒗\mathbf{v} is the internuclear vector that points from the location of SaS_a to SbS_b . θ\theta is the angle between 𝑩\mathbf{B} and 𝒗\mathbf{v} . Credit: Bruce Donald and Anthony Yan.

∥𝑩∥=1(5.32)\|\mathbf{B}\| = 1 \qquad(5.32)

∥𝒗∥=1(5.33)\|\mathbf{v}\| = 1 \qquad(5.33)

D=(K/2)⟨3cos⁡2θ−1⟩(5.34)D = (K/2) \langle 3 \cos^2 \theta - 1 \rangle \qquad(5.34)

=(K/2)⟨3(𝑩⋅𝒗)2−1⟩(5.35)= (K/2) \langle 3(\mathbf{B} \cdot \mathbf{v})^2 - 1 \rangle \qquad(5.35)

=(K/2)⟨3(𝒗⋅𝑩)(𝑩⋅𝒗)−1⟩(5.36)= (K/2) \langle 3(\mathbf{v} \cdot \mathbf{B})(\mathbf{B} \cdot \mathbf{v}) - 1 \rangle \qquad(5.36)

=(K/2)⟨3(𝒗T𝑩)(𝑩T𝒗)−1⟩(5.37)= (K/2) \langle 3(\mathbf{v}^T \mathbf{B})(\mathbf{B}^T \mathbf{v}) - 1 \rangle \qquad(5.37)

=(K/2)⟨3𝒗T(𝑩𝑩T)𝒗−1⟩(5.38)= (K/2) \langle 3\mathbf{v}^T (\mathbf{B}\mathbf{B}^T)\mathbf{v} - 1 \rangle \qquad(5.38)

=(K/2)⟨3𝒗T(𝑩𝑩T)𝒗−𝒗TI𝒗⟩,(5.39)= (K/2) \langle 3\mathbf{v}^T (\mathbf{B}\mathbf{B}^T)\mathbf{v} - \mathbf{v}^T I\mathbf{v} \rangle, \qquad(5.39) where II is the identity matrix.

We now make a physical approximating assumption. Let’s consider our protein to be perfectly rigid. With a rigid protein, we can define a molecular coordinate frame where 𝒗\mathbf{v} is constant, and 𝑩\mathbf{B} varies because the entire ensemble of proteins are tumbling in solution. Since 𝒗\mathbf{v} is constant, we can factor it out:

D=(K/2)𝒗T⟨3(𝑩𝑩T)−I⟩𝒗(5.40)D = (K/2)\mathbf{v}^T \langle 3(\mathbf{B}\mathbf{B}^T) - I \rangle \mathbf{v} \qquad(5.40)

D=(K/2)𝒗TS𝒗(5.41)D = (K/2)\mathbf{v}^T S\mathbf{v} \qquad(5.41)

S≡⟨3(𝑩𝑩T)−I⟩.(5.42)S \equiv \langle 3(\mathbf{B}\mathbf{B}^T) - I \rangle. \qquad(5.42)

SS is known as the Saupe matrix or alignment tensor. SS is a symmetric, real-valued, and traceless matrix. To show that SS is traceless, one only needs to show that Tr(𝑩𝑩T)=1\text{Tr}(\mathbf{B}\mathbf{B}^T) = 1 . Let {𝑩i}\{\mathbf{B}_i\} represent the direction of the magnetic field in the molecular frames of the ensemble of proteins in solution, where i∈{1,2,3,…,N}i \in \{1, 2, 3, \dots, N\} . Then,

⟨𝑩𝑩T⟩=1N∑i=1N𝑩i𝑩iT.(5.43)\langle \mathbf{B}\mathbf{B}^T \rangle = \frac{1}{N} \sum_{i=1}^N \mathbf{B}_i \mathbf{B}_i^T. \qquad(5.43)

Consider the trace of each term:

Tr(𝑩i𝑩iT)=Tr[Bx2BxByBxBzByBxBy2ByBzBzBxBzByBz2]=Bx2+By2+Bz2=1(5.44)\text{Tr}(\mathbf{B}_i \mathbf{B}_i^T) = \text{Tr} \begin{bmatrix} B_x^2 & B_x B_y & B_x B_z \\ B_y B_x & B_y^2 & B_y B_z \\ B_z B_x & B_z B_y & B_z^2 \end{bmatrix} = B_x^2 + B_y^2 + B_z^2 = 1 \qquad(5.44)

Tr⟨𝑩i𝑩iT⟩=1N∑iN1=1(5.45)\text{Tr} \langle \mathbf{B}_i \mathbf{B}_i^T \rangle = \frac{1}{N} \sum_i^N 1 = 1 \qquad(5.45)

Tr⟨3𝑩i𝑩iT−I⟩=3−3=0.(5.46)\text{Tr} \langle 3\mathbf{B}_i \mathbf{B}_i^T - I \rangle = 3 - 3 = 0. \qquad(5.46)

In a homogeneous solution, the distribution of 𝑩\mathbf{B} will be spherically uniform since the protein will be tumbling isotropically in all directions. A consequence of this isotropic tumbling is that SS will become the zero matrix, and the measured RDC effect will be zero. To perform RDC experiments, aligning media are introduced into the solution. The aligning media make the solution anisotropic and cause the proteins to tumble anisotropically. Examples of aligning media include bicelles, phage, and stretched polyacrimide gels. Bicelles are very large flat planar walls of molecules. The wall-like bicelles are packed densely enough that they are roughly parallel, and form a liquid crystal. If the protein is not spherical, then steric collisions with the wall tend to favor some orientations of the protein and disfavor other orientations. As a result, the protein will not tumble isotropically, and the Saupe matrix SS will not be zero. Other aligning media are believed to work in similar ways, although other aligning effects may be due to charge interactions between surface charges in the media and charges on the protein.

Notice that equations (5.39), (5.42), and (5.43) are similar to a covariance matrix. Because the {𝑩i}\{\mathbf{B}_i\} are not zero-meaned, (5.42) is a pseudo covariance matrix with the identity subtracted off.

While the preceding is the most common formalism presented for the Saupe matrix, we next consider a variation in which we do not assume the protein is perfectly rigid. In fact, we are more interested in the structure and dynamics of the protein than in the Saupe matrix SS itself. But the same formalism that describes SS and {𝑩i}\{\mathbf{B}_i\} can be used to characterize the distribution of internuclear bond vectors 𝒗i\mathbf{v}_i due to protein flexibility. More precisely, there is an alignment tensor for 𝒗i\mathbf{v}_i that has exactly the same form as SS . Below, we derive this (well-known) formalism.

First, we need a special kind of matrix multiplication, which is essentially the dot product between two matrices.

Definition 5.3 Given two matrices AA and BB , which are both N×MN \times M , the Frobenius inner product of AA and BB is

A⊙B≡∑i=1N∑j=1MAijBij(5.47)A \odot B \equiv \sum_{i=1}^N \sum_{j=1}^M A_{ij} B_{ij} \quad (5.47) where AijA_{ij} and BijB_{ij} are the ijthij^{\text{th}} component of the respective matrix.

Notice that ⊙\odot is commutative and distributive just like the vector dot product. If AA and BB are square, this can also be written as A⊙B=Tr(A(BT))A \odot B = \text{Tr}(A(B^T)) where Tr(⋅)\text{Tr}(\cdot) is the trace of a matrix. If BB is symmetric, then this reduces to A⊙B=Tr(AB)A \odot B = \text{Tr}(AB) .

Now let’s return to equation (5.39), which precedes our assumption that the protein is rigid. Notice we have a term of the form 𝒗T𝑩𝑩T𝒗\mathbf{v}^T \mathbf{B} \mathbf{B}^T \mathbf{v} . We have

Let G=𝒗𝒗T(5.48)\text{Let } G = \mathbf{v} \mathbf{v}^T \quad (5.48)

Let H=𝑩𝑩T(5.49)\text{Let } H = \mathbf{B} \mathbf{B}^T \quad (5.49)

𝒗T𝑩𝑩T𝒗=𝑩T𝒗𝒗T𝑩=G⊙H.(5.50)\mathbf{v}^T \mathbf{B} \mathbf{B}^T \mathbf{v} = \mathbf{B}^T \mathbf{v} \mathbf{v}^T \mathbf{B} = G \odot H. \quad (5.50)

Furthermore, note that

(𝒗𝒗T)⊙I=[vx2vxvyvxvzvyvxvy2vyvzvzvxvzvyvz2]⊙[111]=Tr(𝒗𝒗T)=vx2+vy2+vz2=1(5.51)(\mathbf{v}\mathbf{v}^T) \odot I = \begin{bmatrix} v_x^2 & v_x v_y & v_x v_z \\ v_y v_x & v_y^2 & v_y v_z \\ v_z v_x & v_z v_y & v_z^2 \end{bmatrix} \odot \begin{bmatrix} 1 & & \\ & 1 & \\ & & 1 \end{bmatrix} = \text{Tr}(\mathbf{v}\mathbf{v}^T) = v_x^2 + v_y^2 + v_z^2 = 1 \quad (5.51) because ∥𝒗∥=1\|\mathbf{v}\| = 1 .

Using equations (5.50) and (5.51), one can factor equation (5.39) into two parts:

D=(K/2)⟨3𝒗iT(𝑩i𝑩iT)𝒗i−1⟩(5.52)D = (K/2) \langle 3\mathbf{v}_i^T (\mathbf{B}_i \mathbf{B}_i^T) \mathbf{v}_i - 1 \rangle \quad (5.52)

=(K/6)⟨9𝒗iT(𝑩i𝑩iT)𝒗i−3⟩(5.53)= (K/6) \langle 9\mathbf{v}_i^T (\mathbf{B}_i \mathbf{B}_i^T) \mathbf{v}_i - 3 \rangle \quad (5.53)

=(K/6)⟨9(𝑩i𝑩iT)⊙(𝒗i𝒗iT)−3⟩(5.54)= (K/6) \langle 9(\mathbf{B}_i \mathbf{B}_i^T) \odot (\mathbf{v}_i \mathbf{v}_i^T) - 3 \rangle \quad (5.54)

=(K/6)⟨9(𝑩i𝑩iT)⊙(𝒗i𝒗iT)−3−3+3⟩(5.55)= (K/6) \langle 9(\mathbf{B}_i \mathbf{B}_i^T) \odot (\mathbf{v}_i \mathbf{v}_i^T) - 3 - 3 + 3 \rangle \quad (5.55)

=(K/6)⟨9(𝑩i𝑩iT)⊙(𝒗i𝒗iT)−3(𝑩i𝑩iT)⊙I−3I⊙(𝒗i𝒗iT)+3⟩(5.56)= (K/6) \langle 9(\mathbf{B}_i \mathbf{B}_i^T) \odot (\mathbf{v}_i \mathbf{v}_i^T) - 3(\mathbf{B}_i \mathbf{B}_i^T) \odot I - 3I \odot (\mathbf{v}_i \mathbf{v}_i^T) + 3 \rangle \quad (5.56)

=(K/6)⟨(3𝑩i𝑩iT−I)⊙(3𝒗i𝒗iT−I)⟩(5.57)= (K/6) \langle (3\mathbf{B}_i \mathbf{B}_i^T - I) \odot (3\mathbf{v}_i \mathbf{v}_i^T - I) \rangle \quad (5.57)

D=(K/6)⟨E⊙F⟩(5.58)D = (K/6) \langle E \odot F \rangle \quad (5.58) where

E=3𝑩i𝑩iT−I(5.59)E = 3\mathbf{B}_i \mathbf{B}_i^T - I \quad (5.59)

F=3𝒗i𝒗iT−I.(5.60)F = 3\mathbf{v}_i \mathbf{v}_i^T - I. \quad (5.60)

At this point, we assume that the overall tumbling of the protein (which determines EE) is uncorrelated with the flexing of the protein (which determines FF in the molecular frame). If two variables are uncorrelated, then the expectation of their product is the product of their expectations:

D=(K/6)⟨E⟩⊙⟨F⟩=(K/6)S⊙T.(5.61)D = (K/6) \langle E \rangle \odot \langle F \rangle = (K/6) S \odot T. \quad (5.61)

We define the Saupe matrix S≡⟨E⟩=⟨3𝑩i𝑩iT−I⟩S \equiv \langle E \rangle = \langle 3\mathbf{B}_i \mathbf{B}_i^T - I \rangle .

We define the alignment tensor for 𝒗i\mathbf{v}_i to be T≡⟨F⟩=⟨3𝒗i𝒗iT−I⟩T \equiv \langle F \rangle = \langle 3\mathbf{v}_i \mathbf{v}_i^T - I \rangle .

The literature often defines S≡(K/2)⟨E⟩S \equiv (K/2) \langle E \rangle . Different constants may be found in the definition of TT according to different primary researchers.

If the 𝒗i\mathbf{v}_i are identical (i.e., the protein is rigid) in the molecular frame, then mathematically equation (5.61) reduces to equation (5.41).

Both the Saupe matrix SS and 𝒗i\mathbf{v}_i ’s alignment tensor have the same form. For protein flexibility (dynamics), we would like to know the distribution of {𝒗i}\{\mathbf{v}_i\} where the 𝒗i\mathbf{v}_i are expressed in the molecular frame. (Because the protein is flexing, a molecular frame may not be well defined; however, it may be reasonably defined.)

It turns out that finding TT is often an underconstrained problem. However, using a procedure described by Tolman [1], it seems possible to estimate TT . Tolman’s procedure requires RDCs in five aligning media which is considered to be a high number of media by current experimental standards. Also, Tolman’s problem is underconstrained, so he is required to make some reasonable assumptions, which can be rationalized. Tolman’s procedure can make reasonable (or at least plausible) estimates of TT .

Once we have TT , we would like to characterize the distribution {𝒗i}\{\mathbf{v}_i\} in the molecular frame. This, in turn, will help us understand protein flexibility.

Now that we have presented the alignment tensor formalism, and motivated our interest in it, we proceed to the next section where we describe in detail the relationship between alignment tensors and covariance matrices.

RDCs and PCA

In this section we relate RDCs to PCA by showing the connection between the alignment tensor and covariance matrices. To begin, let’s compare the two, side by side.

Let {𝒗i}\{\mathbf{v}_i\} be an arbitrary set of unit vectors.

𝒗i∈ℝm,i∈{1,2,3,…,m} and ∥𝒗i∥=1(5.62)\mathbf{v}_i \in \mathbb{R}^m, i \in \{1, 2, 3, \dots, m\} \text{ and } \|\mathbf{v}_i\| = 1 \quad (5.62)

⟨𝒗i⟩=1N∑i=1N𝒗i(5.63)\langle \mathbf{v}_i \rangle = \frac{1}{N} \sum_{i=1}^N \mathbf{v}_i \quad (5.63)

Covariance Matrix: MP=1N∑i=1N(𝒗i−⟨𝒗i⟩)(𝒗i−⟨𝒗i⟩)T(5.64)\text{Covariance Matrix: } M_P = \frac{1}{N} \sum_{i=1}^N (\mathbf{v}_i - \langle \mathbf{v}_i \rangle)(\mathbf{v}_i - \langle \mathbf{v}_i \rangle)^T \quad (5.64)

Alignment Tensor: T=(K/6)1N∑i=1N(3𝒗𝒗T−I)=(K/6)(3Q−I)(5.65)\text{Alignment Tensor: } T = (K/6) \frac{1}{N} \sum_{i=1}^N (3\mathbf{v}\mathbf{v}^T - I) = (K/6)(3Q - I) \quad (5.65)

where Q≡1N∑i=1N𝒗𝒗T is a pseudo covariance matrix.(5.66)\text{where } Q \equiv \frac{1}{N} \sum_{i=1}^N \mathbf{v}\mathbf{v}^T \text{ is a pseudo covariance matrix.} \quad (5.66)

Our goal is to get as much information about the distribution {𝒗i}\{\mathbf{v}_i\} as possible. In particular, we would like to perform PCA on {𝒗i}\{\mathbf{v}_i\} , or if we can’t, we would like to have as much statistics on {𝒗i}\{\mathbf{v}_i\} as possible.

The two differences between MPM_P and TT are that the outer products in TT are not from zero-meaned vectors, and the final result has the identity subtracted off (The identity can be pulled out of the summation because it is constant, so its average is itself.) As we shall see, subtracting the identity is essentially no problem, but the lack of zero-meaning of the data is a big problem.

First, let us address the subtraction of the identity. It is trivial to see that TT has the same eigenvectors as QQ , and the eigenvalues of TT have a simple relationship. This is because, if 𝒙\mathbf{x} is an eigenvector of QQ , then multiplication by either QQ or the identity II , does not change the direction of 𝒙\mathbf{x} :

Q𝒙=(λQ)𝒙(5.67)Q\mathbf{x} = (\lambda_Q)\mathbf{x} \quad (5.67)

⇔T𝒙=(K/6)(3Q−I)𝒙=(K/6)(3Q𝒙−𝒙)(5.68)\iff T\mathbf{x} = (K/6)(3Q - I)\mathbf{x} = (K/6)(3Q\mathbf{x} - \mathbf{x}) \quad (5.68)

⇔T𝒙=(K/6)(3λQ𝒙−𝒙)(5.69)\iff T\mathbf{x} = (K/6)(3\lambda_Q\mathbf{x} - \mathbf{x}) \quad (5.69)

⇔T𝒙=(K/6)(3λQ−1)𝒙(5.70)\iff T\mathbf{x} = (K/6)(3\lambda_Q - 1)\mathbf{x} \quad (5.70)

⇔T𝒙=(λT)𝒙 where λT=(K/6)(3λQ−1).(5.71)\iff T\mathbf{x} = (\lambda_T)\mathbf{x} \text{ where } \lambda_T = (K/6)(3\lambda_Q - 1). \quad (5.71)

So if we know TT and KK , we can solve for the pseudo covariance matrix QQ . In practice, we know TT only up to a scale factor, in which case we can solve for QQ up to a scale factor.

Given TT , we know the pseudo covariance matrix for {𝒗i}\{\mathbf{v}_i\} . What is the relationship between the pseudo covariance matrices and true covariance matrices? As we will show later, the directional variance of a pseudo covariance matrix is an upper bound on the eigenvalues of the true covariance matrix (directional variances of the true covariance matrix). To prove this, we first show that the pseudo covariance matrix QQ of {𝒗i}\{\mathbf{v}_i\} is actually the true covariance matrix of a different point set. Then, we will show that the directional variance of that different point set is always greater than or equal to the directional variance of the original point set {𝒗i}\{\mathbf{v}_i\} . Finally, we will show that if the mean of {𝒗i}\{\mathbf{v}_i\} were known, then we could convert the pseudo covariance matrix QQ into the true covariance matrix for {𝒗i}\{\mathbf{v}_i\} :

Let V+={𝒗i} where 𝒗i∈ℝm,i∈{1,2,3,…,m} and ∥𝒗i∥=1(5.72)\text{Let } V^+ = \{\mathbf{v}_i\} \text{ where } \mathbf{v}_i \in \mathbb{R}^m, i \in \{1, 2, 3, \dots, m\} \text{ and } \|\mathbf{v}_i\| = 1 \quad (5.72)

Let V−={−𝒗i∣𝒗i∈V+}(5.73)\text{Let } V^- = \{-\mathbf{v}_i \mid \mathbf{v}_i \in V^+\} \quad (5.73)

Let W=V+∪V−(5.74)\text{Let } W = V^+ \cup V^- \quad (5.74)

Let Q=1N∑i𝒗i𝒗iT pseudo covariance matrix.(5.75)\text{Let } Q = \frac{1}{N} \sum_i \mathbf{v}_i \mathbf{v}_i^T \text{ pseudo covariance matrix.} \quad (5.75)

We now claim that QQ is the true covariance matrix of W=V+∪V−W = V^+ \cup V^- . First, we show that WW is zero-meaned:

12N∑𝒘∈W𝒘=1N∑i=1N(𝒗i−𝒗i)=0.(5.76)\frac{1}{2N} \sum_{\mathbf{w} \in W} \mathbf{w} = \frac{1}{N} \sum_{i=1}^N (\mathbf{v}_i - \mathbf{v}_i) = 0. \quad (5.76)

Next, we compute the true covariance matrix of WW and show it is equal to QQ :

MW=12N∑𝒘∈W𝒘𝒘T=12[1N∑𝒗i𝒗iT+1N∑(−𝒗i)(−𝒗iT)](5.77)M_W = \frac{1}{2N} \sum_{\mathbf{w} \in W} \mathbf{w} \mathbf{w}^T = \frac{1}{2} \left[ \frac{1}{N} \sum \mathbf{v}_i \mathbf{v}_i^T + \frac{1}{N} \sum (-\mathbf{v}_i)(-\mathbf{v}_i^T) \right] \quad (5.77)

=12[1N∑𝒗i𝒗iT+1N∑𝒗i𝒗iT](5.78)= \frac{1}{2} \left[ \frac{1}{N} \sum \mathbf{v}_i \mathbf{v}_i^T + \frac{1}{N} \sum \mathbf{v}_i \mathbf{v}_i^T \right] \quad (5.78)

=12[2Q](5.79)= \frac{1}{2} [2Q] \quad (5.79)

=Q.(5.80)= Q. \quad (5.80)

Intuitively, we have performed PCA not on the original data V+={𝒗i}V^+ = \{\mathbf{v}_i\} , but on “the original data plus its reflection,” namely, PCA on W=V+∪V−W = V^+ \cup V^- . See figure 5.5 for an example in ℝ2\mathbb{R}^2 .

Next, we show that the directional variance of V+∪V−V^+ \cup V^- is always greater than or equal to the directional variance of V+V^+ . To do this, we start with zero-meaned data and show that if the data is translated in any direction, the directional standard deviation will remain the same or increase. In fact, the directional variance along the direction of translation will always increase:

Figure 5.5 PCA of W=V+∪V−W = V^+ \cup V^- in ℝ2\mathbb{R}^2 . The x’s mark the location of data points 𝒗i∈V+\mathbf{v}_i \in V^+ . The shaded ellipse represents PCA on V+V^+ and is the level set of the associated Gaussian for V+V^+ . The large unshaded ellipse represents PCA on W=V+∪V−W = V^+ \cup V^- and is the level set of the associated Gaussian for WW . The means of the three sets WW , V+V^+ , and V−V^- are marked by dark filled circles. Credit: Bruce Donald and Anthony Yan.

Let X0={𝒙i}X_0 = \{\mathbf{x}_i\} where

𝒙i∈ℝm and i∈{1,2,3,…,N}(5.81)\mathbf{x}_i \in \mathbb{R}^m \text{ and } i \in \{1, 2, 3, \dots, N\} \quad (5.81)

By assumption, X0X_0 is zero-meaned:

⟨𝒙i⟩=0(5.82)\langle \mathbf{x}_i \rangle = 0 \quad (5.82)

Let 𝒉∈ℝm\mathbf{h} \in \mathbb{R}^m be an arbitrary offset. (5.83)

Let X𝒉+={𝒚i∣𝒚i=𝒙i+𝒉}(5.84)\text{Let } X_{\mathbf{h}}^+ = \{\mathbf{y}_i \mid \mathbf{y}_i = \mathbf{x}_i + \mathbf{h}\} \quad (5.84) where 𝒙i∈X0\mathbf{x}_i \in X_0

Let X𝒉−={−𝒚i∣𝒚i∈X𝒉+}.(5.85)\text{Let } X_{\mathbf{h}}^- = \{-\mathbf{y}_i \mid \mathbf{y}_i \in X_{\mathbf{h}}^+\}. \quad (5.85)

Let W𝒉=X𝒉+∪X𝒉−.(5.86)\text{Let } W_{\mathbf{h}} = X_{\mathbf{h}}^+ \cup X_{\mathbf{h}}^-. \quad (5.86)

We claim that for any direction 𝒖\mathbf{u} , the directional variance of W𝒉=X𝒉+∪X𝒉−W_{\mathbf{h}} = X_{\mathbf{h}}^+ \cup X_{\mathbf{h}}^- is greater than or equal to the directional variance of X0X_0 . First, let’s consider the covariance matrix of X0X_0 and W𝒉W_{\mathbf{h}} :

MX0=1N∑i=1N𝒙i𝒙iT(5.87)M_{X_0} = \frac{1}{N} \sum_{i=1}^N \mathbf{x}_i \mathbf{x}_i^T \quad (5.87)

MW𝒉=12N[∑i=1N(𝒙i+𝒉)(𝒙i+𝒉)T+∑i=1N[−(𝒙i+𝒉)][−(𝒙i+𝒉)]T](5.88)M_{W_{\mathbf{h}}} = \frac{1}{2N} \left[ \sum_{i=1}^N (\mathbf{x}_i + \mathbf{h})(\mathbf{x}_i + \mathbf{h})^T + \sum_{i=1}^N [-(\mathbf{x}_i + \mathbf{h})][-(\mathbf{x}_i + \mathbf{h})]^T \right] \quad (5.88)

=22N[∑i=1N(𝒙i+𝒉)(𝒙i+𝒉)T](5.89)= \frac{2}{2N} \left[ \sum_{i=1}^N (\mathbf{x}_i + \mathbf{h})(\mathbf{x}_i + \mathbf{h})^T \right] \quad (5.89)

=1N∑i=1N(𝒙i𝒙iT+𝒙i𝒉T+𝒉𝒙iT+𝒉𝒉T)(5.90)= \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{x}_i \mathbf{h}^T + \mathbf{h} \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) \quad (5.90)

=1N∑i=1N(𝒙i𝒙iT+𝒉𝒉T)+1N∑i=1N(𝒙i⋅𝒉+𝒉⋅𝒙i)(5.91)= \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \cdot \mathbf{h} + \mathbf{h} \cdot \mathbf{x}_i) \quad (5.91)

=1N∑i=1N(𝒙i𝒙iT+𝒉𝒉T)+2𝒉⋅[1N∑i=1N𝒙i](5.92)= \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + 2\mathbf{h} \cdot \left[ \frac{1}{N} \sum_{i=1}^N \mathbf{x}_i \right] \quad (5.92)

=1N∑i=1N(𝒙i𝒙iT+𝒉𝒉T)+2𝒉⋅⟨𝒙i⟩(5.93)= \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + 2\mathbf{h} \cdot \langle \mathbf{x}_i \rangle \quad (5.93)

=1N∑i=1N(𝒙i𝒙iT+𝒉𝒉T)(5.94)= \frac{1}{N} \sum_{i=1}^N (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) \quad (5.94)

=1N∑i=1N𝒙i𝒙iT+1N∑i=1N𝒉𝒉T(5.95)= \frac{1}{N} \sum_{i=1}^N \mathbf{x}_i \mathbf{x}_i^T + \frac{1}{N} \sum_{i=1}^N \mathbf{h} \mathbf{h}^T \quad (5.95)

=MX0+NN𝒉𝒉T(5.96)= M_{X_0} + \frac{N}{N} \mathbf{h} \mathbf{h}^T \quad (5.96)

MWh=MX0+𝒉𝒉T.(5.97)M_{W_h} = M_{X_0} + \mathbf{h} \mathbf{h}^T. \quad (5.97)

Given the covariance matrices for X0X_0 and Xh+∪Xh−X_h^+ \cup X_h^- , we can now compare their directional variances:

σ𝒖2(X0)=𝒖T(MX0)𝒖(5.98)\sigma_{\mathbf{u}}^2(X_0) = \mathbf{u}^T (M_{X_0}) \mathbf{u} \quad (5.98)

σ𝒖2(Wh)=𝒖T(MX0+𝒉𝒉T)𝒖(5.99)\sigma_{\mathbf{u}}^2(W_h) = \mathbf{u}^T (M_{X_0} + \mathbf{h} \mathbf{h}^T) \mathbf{u} \quad (5.99)

σ𝒖2(Wh)=𝒖T(MX0)𝒖+𝒖T𝒉𝒉T𝒖(5.100)\sigma_{\mathbf{u}}^2(W_h) = \mathbf{u}^T (M_{X_0}) \mathbf{u} + \mathbf{u}^T \mathbf{h} \mathbf{h}^T \mathbf{u} \quad (5.100)

σ𝒖2(Wh)=σ𝒖2(X0)+(𝒖⋅𝒉)(𝒉⋅𝒖)(5.101)\sigma_{\mathbf{u}}^2(W_h) = \sigma_{\mathbf{u}}^2(X_0) + (\mathbf{u} \cdot \mathbf{h})(\mathbf{h} \cdot \mathbf{u}) \quad (5.101)

σ𝒖2(Wh)=σ𝒖2(X0)+(𝒖⋅𝒉)2.(5.102)\sigma_{\mathbf{u}}^2(W_h) = \sigma_{\mathbf{u}}^2(X_0) + (\mathbf{u} \cdot \mathbf{h})^2. \quad (5.102)

Both WhW_h and X0X_0 are positive semidefinite, and (𝒖⋅𝒉)2(\mathbf{u} \cdot \mathbf{h})^2 is also always greater than or equal to zero. So all the terms of equation (5.102) are non-negative. As a result, we can conclude that

σ𝒖2(Wh)=σ𝒖2(Xh+∪Xh−)≥σ𝒖2(X0).(5.103)\sigma_{\mathbf{u}}^2(W_h) = \sigma_{\mathbf{u}}^2(X_h^+ \cup X_h^-) \geq \sigma_{\mathbf{u}}^2(X_0). \quad (5.103)

Intuitively, the nonzero meaned pseudo covariance matrix always has a larger direction variance than the zero-meaned covariance matrix because the nonzero-meaned case is “stretched apart” by its shadow. (see figure 5.5).

So given MWhM_{W_h} (which is equal to MXh+M_{X_h^+}), we have upper bounds on the directional variances of X0X_0 . In the event that we know 𝒉\mathbf{h} (which is the mean of Xh+X_h^+), we can use equation (5.97) to compute the original covariance matrix MX0M_{X_0} from the pseudo covariance matrix MXh+M_{X_h^+} because MWh=MXh+M_{W_h} = M_{X_h^+} . This would allow us to perform PCA on X0X_0 even though we were only given the pseudo covariance matrix.

One might wonder if equations (5.97) and (5.102) can be used to solve for MX0M_{X_0} by minimizing [σ𝒖2(Wh)−(𝒖⋅𝒉)2][\sigma_{\mathbf{u}}^2(W_h) - (\mathbf{u} \cdot \mathbf{h})^2] . Intuitively, the idea is to find the translation 𝒉\mathbf{h} that minimizes the directional variances as much as possible. Unfortunately, this is not the case. The intuition is reasonable, however, we cannot perform all the required steps.

Suppose we are given a pseudo covariance matrix MAM_A of some point set A={𝒙i}A = \{\mathbf{x}_i\} , which is arbitrary (not zero-meaned). Now we wish to compute the pseudo covariance matrix of AhA_h , which is the translation of AA :

Ah={(𝒙i+𝒉)∣𝒙i∈A}.(5.104)A_h = \{(\mathbf{x}_i + \mathbf{h}) \mid \mathbf{x}_i \in A\}. \quad (5.104)

We are given only MAM_A and 𝒉\mathbf{h} . Can we compute MA𝒉M_{A\mathbf{h}} ? The answer is no:

MA𝒉=1N∑i(𝒙i+𝒉)(𝒙i+𝒉)T(5.105)M_{A\mathbf{h}} = \frac{1}{N} \sum_i (\mathbf{x}_i + \mathbf{h})(\mathbf{x}_i + \mathbf{h})^T \quad (5.105)

MA𝒉=1N∑i(𝒙i𝒙iT+𝒉𝒉T)+1N∑i[(𝒙i⋅𝒉)+(𝒉⋅𝒙i)](5.106)M_{A\mathbf{h}} = \frac{1}{N} \sum_i (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + \frac{1}{N} \sum_i [(\mathbf{x}_i \cdot \mathbf{h}) + (\mathbf{h} \cdot \mathbf{x}_i)] \quad (5.106)

MA𝒉=1N∑i(𝒙i𝒙iT+𝒉𝒉T)+21N∑i(𝒉⋅𝒙i)(5.107)M_{A\mathbf{h}} = \frac{1}{N} \sum_i (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + 2 \frac{1}{N} \sum_i (\mathbf{h} \cdot \mathbf{x}_i) \quad (5.107)

MA𝒉=1N∑i(𝒙i𝒙iT+𝒉𝒉T)+2𝒉⋅(1N∑i𝒙i)(5.108)M_{A\mathbf{h}} = \frac{1}{N} \sum_i (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + 2\mathbf{h} \cdot \left( \frac{1}{N} \sum_i \mathbf{x}_i \right) \quad (5.108)

MA𝒉=1N∑i(𝒙i𝒙iT+𝒉𝒉T)+2𝒉⋅⟨𝒙i⟩.(5.109)M_{A\mathbf{h}} = \frac{1}{N} \sum_i (\mathbf{x}_i \mathbf{x}_i^T + \mathbf{h} \mathbf{h}^T) + 2\mathbf{h} \cdot \langle \mathbf{x}_i \rangle. \quad (5.109)

So we can see that we need to know the average of {𝒙i}\{\mathbf{x}_i\} to solve for MA𝒉M_{A\mathbf{h}} . In case that the data is already zero-meaned, then ⟨𝒙i⟩=0\langle \mathbf{x}_i \rangle = 0 and the above reduces to equation (5.97).

Conclusions and Future Work

Though we don’t know the average ⟨𝒙i⟩\langle \mathbf{x}_i \rangle from the RDC data, there may be reasonable ways to estimate it based on the area of application. For example, if the protein’s X-ray or NMR structure is known, then the internuclear vectors of the known structure might be reasonable estimates of the average ⟨𝒙i⟩\langle \mathbf{x}_i \rangle . Given such an estimate, it becomes possible to estimate MX0M_{X_0} and perform PCA on {𝒙i}\{\mathbf{x}_i\} .

In some cases, the vectors {𝒙i}\{\mathbf{x}_i\} may be closely clustered on the unit sphere. If their directional variances are much less than unity, then they can be considered to be tightly clustered. In this case, the principle eigenvector of the pseudo covariance matrix may be a reasonable approximation for the average ⟨𝒙i⟩\langle \mathbf{x}_i \rangle . However, normally in RDC data, the Saupe matrix is nearly zero because the vast majority of proteins are tumbling isotropically in solution. Only a tiny fraction of proteins interact with the aligning media and tumble anisotropically. As a result, we know that the directional variance will be almost the same in all directions, and in fact, the mean will be close to zero. In a sense, we are fortunate that the Saupe matrix subtracts the identity from the pseudo covariance, because if this were not the case, then the eigenvalues of the Saupe matrix would be nearly identical. That, in turn, would make computation of the eigenvectors of the Saupe matrix less numerically stable. If two eigenvalues are equal, their directions must lie in the plane spanned by both of them, but their directions are otherwise unrestricted in that plane. Thus, if the two eigenvalues are nearly equal, their directions are nearly unconstrained in the plane that they span.

Finally, it may be possible to add information or assumptions that result in a lower bound on the directional variances.

Appendix

In this appendix we consider how to compute PCA efficiently when M≫N≫1M \gg N \gg 1 , where MM is the dimension of our data, and NN is the number of data points. We are given 𝒙i∈ℝM\mathbf{x}_i \in \mathbb{R}^M with i∈{1,2,3,…,N}i \in \{1, 2, 3, \dots, N\} . We would like to compute PCA on {𝒙i}\{\mathbf{x}_i\} while not being overwhelmed by MM .

First, construct a data matrix where the columns are the data vectors 𝒙i\mathbf{x}_i :

A≡[𝒙1|𝒙2|𝒙3|⋯|𝒙M].(5.110)A \equiv \begin{bmatrix} \mathbf{x}_1 & | & \mathbf{x}_2 & | & \mathbf{x}_3 & | & \cdots & | & \mathbf{x}_M \end{bmatrix}. \quad (5.110)

We can compute the covariance matrix MA=AATM_A = AA^T . But, MAM_A is now M×MM \times M , which is huge because M≫N≫1M \gg N \gg 1 . Working with MAM_A is computationally infeasible. Notice, though, that NN points in any dimension live in a subspace whose dimension is at most N−1N - 1 , which is much less than MM . (For example, three points determine a two-dimensional plane, no matter what dimension the points live in themselves.) So it seems natural to suspect that we can work with a matrix that is N×NN \times N rather than M×MM \times M .

The matrix ATAA^T A is N×NN \times N . What is its relationship to AATAA^T ? Notice that both ATAA^T A and AATAA^T are symmetric real matrices. So both will have orthogonal eigenvectors. Suppose that we have an eigenvector of AATAA^T :

AAT𝒗=λ𝒗.(5.111)AA^T \mathbf{v} = \lambda \mathbf{v}. \quad (5.111)

Then we can multiply both sides by ATA^T :

ATAAT𝒗=λAT𝒗(5.112)A^T A A^T \mathbf{v} = \lambda A^T \mathbf{v} \quad (5.112)

ATA(AT𝒗)=λ(AT𝒗).(5.113)A^T A (A^T \mathbf{v}) = \lambda (A^T \mathbf{v}). \quad (5.113)

So AT𝒗A^T \mathbf{v} is an eigenvector of ATAA^T A with the same eigenvalue λ\lambda .

Similarly, suppose we have an eigenvector of ATAA^T A :

ATA𝒖=γ𝒖(5.114)A^T A \mathbf{u} = \gamma \mathbf{u} \quad (5.114)

AATA𝒖=γA𝒖(5.115)AA^T A \mathbf{u} = \gamma A \mathbf{u} \quad (5.115)

AAT(A𝒖)=γ(A𝒖).(5.116)AA^T (A \mathbf{u}) = \gamma (A \mathbf{u}). \quad (5.116)

(5.117)(5.117)

So A𝒖A \mathbf{u} is also an eigenvector of AATAA^T with the same eigenvalue γ\gamma .

As a result, we can solve the eigen-problem for the smaller matrix ATAA^T A and then transform the eigenvalues and eigenvectors into solutions of the eigen-problem of the larger matrix AATAA^T . This can save a tremendous amount of computation, and allows PCA to be feasible in practice.

Reference

[1] J. R. Tolman. A novel approach to the retrieval of structural and dynamic information from residual dipolar couplings using several oriented media in biomolecular NMR spectroscopy. Journal American Chemical Society, 124 (2002): 12020–12030.