For correlated random variables [math]X[/math] and [math]Y[/math] it is natural to ask for the expected value for [math]X[/math] given [math]Y[/math]. For example, Galton calculated the expected value of the height of a son given the height of the father. He used this to show that tall men can be expected to have sons who are less tall on the average. Similarly, students who do very well on one exam can be expected to do less well on the next exam, and so forth. This is called regression on the mean. To define this conditional expected value, we first define a conditional density of [math]X[/math] given [math]Y = y[/math] by

where [math]f_{X,Y}(x,y)[/math] is the joint density of [math]X[/math] and [math]Y[/math], and [math]f_Y[/math] is the density for [math]Y[/math]. Then the conditional expected value of [math]X[/math] given [math]Y[/math] is

For the normal density in Exercise, show that the conditional density of [math]f_{X|Y}(x|y)[/math] is normal with mean [math]\rho y[/math] and variance [math]1 - \rho^2[/math]. From this we see that if [math]X[/math] and [math]Y[/math] are positively correlated [math](0 \lt \rho \lt 1)[/math], and if [math]y \gt E(Y)[/math], then the expected value for [math]X[/math] given [math]Y = y[/math] will be less than [math]y[/math] (i.e., we have regression on the mean).

A point [math]Y[/math] is chosen at random from [math][0,1][/math]. A second point [math]X[/math] is then chosen from the interval [math][0,Y][/math]. Find the density for [math]X[/math]. Hint: Calculate [math]f_{X|Y}[/math] as in Exercise and then use

Can you also derive your result geometrically?

Let [math]X[/math] and [math]V[/math] be two standard normal random variables. Let [math]\rho[/math] be a real number between -1 and 1.

• Let [math]Y = \rho X + \sqrt{1 - \rho^2} V[/math]. Show that [math]E(Y) = 0[/math] and [math]Var(Y) = 1[/math]. We shall see later (see Example and Example), that the sum of two independent normal random variables is again normal. Thus, assuming this fact, we have shown that [math]Y[/math] is standard normal.
• Using Exercises Exercise and Exercise, show that the correlation of [math]X[/math] and [math]Y[/math] is [math]\rho[/math].
• In Exercise Exercise, the joint density function [math]f_{X,Y}(x, y)[/math] for the random variable [math](X, Y)[/math] is given. Now suppose that we want to know the set of points [math](x, y)[/math] in the [math]xy[/math]-plane such that [math]f_{X,Y}(x, y) = C[/math] for some constant [math]C[/math]. This set of points is called a set of constant density. Roughly speaking, a set of constant density is a set of points where the outcomes [math](X, Y)[/math] are equally likely to fall. Show that for a given [math]C[/math], the set of points of constant density is a curve whose equation is
  [[math]] x^2 - 2\rho x y + y^2 = D\ , [[/math]]
  where [math]D[/math] is a constant which depends upon [math]C[/math]. (This curve is an ellipse.)
• One can plot the ellipse in part (c) by using the parametric equations
  [[math]] \begin{eqnarray*} x & = & \frac {r\cos\theta}{\sqrt{2(1 - \rho)}} + \frac {r\sin\theta}{\sqrt{2(1 + \rho)}}\ , \\ y & = & \frac {r\cos\theta}{\sqrt{2(1 - \rho)}} - \frac {r\sin\theta}{\sqrt{2(1 + \rho)}}\ . \end{eqnarray*} [[/math]]
  Write a program to plot 1000 pairs [math](X, Y)[/math] for [math]\rho = -1/2, 0, 1/2[/math]. For each plot, have your program plot the above parametric curves for [math]r = 1, 2, 3[/math].

Following Galton, let us assume that the fathers and sons have heights that are dependent normal random variables. Assume that the average height is 68 inches, standard deviation is 2.7 inches, and the correlation coefficient is .5 (see Exercises Exercise and Exercise). That is, assume that the heights of the fathers and sons have the form [math]2.7X + 68[/math] and [math]2.7Y + 68[/math], respectively, where [math]X[/math] and [math]Y[/math] are correlated standardized normal random variables, with correlation coefficient .5.

• What is the expected height for the son of a father whose height is 72 inches?
• Plot a scatter diagram of the heights of 1000 father and son pairs. Hint: You can choose standardized pairs as in Exercise and then plot [math](2.7X + 68, 2.7Y + 68)[/math].

When we have pairs of data [math](x_i,y_i)[/math] that are outcomes of the pairs of dependent random variables [math]X[/math], [math]Y[/math] we can estimate the coorelation coefficient [math]\rho[/math] by

where [math]\bar x[/math] and [math]\bar y[/math] are the sample means for [math]X[/math] and [math]Y[/math], respectively, and [math]s_X[/math] and [math]s_Y[/math] are the sample standard deviations for [math]X[/math] and [math]Y[/math] (see Exercise). Write a program to compute the sample means, variances, and correlation for such dependent data. Use your program to compute these quantities for Galton's data on heights of parents and children given in Appendix B.

Plot the equal density ellipses as defined in Exercise for [math]r = 4[/math], 6, and 8, and on the same graph print the values that appear in the table at the appropriate points. For example, print 12 at the point [math](70.5,68.2)[/math], indicating that there were 12 cases where the parent's height was 70.5 and the child's was 68.12. See if Galton's data is consistent with the equal density ellipses.

(from Hamming^{[Notes 1]}) Suppose you are standing on the bank of a straight river.

• Choose, at random, a direction which will keep you on dry land, and walk 1 km in that direction. Let [math]P[/math] denote your position. What is the expected distance from [math]P[/math] to the river?
• Now suppose you proceed as in part (a), but when you get to [math]P[/math], you pick a random direction (from among all directions) and walk 1 km. What is the probability that you will reach the river before the second walk is completed?

Notes

(from Hamming^{[Notes 1]}) A game is played as follows: A random number [math]X[/math] is chosen uniformly from [math][0, 1][/math]. Then a sequence [math]Y_1,Y_2,\ldots[/math] of random numbers is chosen independently and uniformly from [math][0, 1][/math]. The game ends the first time that [math]Y_i~ \gt ~X[/math]. You are then paid [math](i-1)[/math] dollars. What is a fair entrance fee for this game?

Notes

A long needle of length [math]L[/math] much bigger than 1 is dropped on a grid with horizontal and vertical lines one unit apart. Show that the average number [math]a[/math] of lines crossed is approximately