The concept of a normal population seems ubiquitous in science and life in general. Our data are often validated with means and standard deviations based on the assumption that the normal curve applies or that the data can be transformed to approximate a normal distribution.
I was studying some sites in Wisconsin that had 14C dates associated with them, and I was trying to determine if the dates were sufficiently alike that I could accept the null hypothesis that they were of the same age and that the reported difference could have arisen by chance. I was also going to calibrate the dates with CALIB3c (Reimer, 1994), and that program has a null test based on the chi- square criterion.
But more than a simple null test, I wanted to estimate the extent to which the populations overlapped. One site at Valders quarry had the date 12,965 ñ 200 yr B.P. The event-of-interest at Devils Lake was bracketed by the three dates: 12,260±115 yr B.P., 12,520±160 yr B.P., and 12,880±125 yr B.P. The mean of the three Devils Lake dates is 12550±233 yr B.P. (mean of the three dates ñ the square root of the sum of the squares of the dates' standard errors).
Assuming that the two sites' dates are normally distributed with means 12965±200 yr B.P. and 12550±233 yr B.P., how much do the two distributions overlap? I thought about this a good deal, and I even considered getting the proportions by plotting the distributions to scale on paper, cutting out the parts, and weighing them. But then I recalled the idea of circular normal probability and the normal joint-probability density function that I had once used in calculating confidence limits for microfossil concentration measurements using samples spiked with marker grains; the details can be found in Maher (1981).
The basic reasoning and proof involves coding and standardizing the two populations so that they can be plotted in units of their respective standard deviations that is, in Z units. This can be done for any value of a variable by subtracting the population's mean and dividing the result by the population's standard deviation. (Of course, any Z unit can be transformed back into the original units by multiplying it by the population's standard deviation and adding its mean.)
Fig. 1 shows the situation for the two 14C dates. The one with the larger mean has been standardized and plotted at the left side; the date with the smaller mean is plotted at the top. The standard normal joint-probability density function defines a bell-shaped surface lying over the (x, y) plane, and centered at the middle of the graph. The joint-probability density (JPD) decreases away from the center; 0.9998 of the JPD occurs within 4.1 Z units of the center, and that limit is shown by the large dashed circle. We will ignore the trivial probability lying outside that circle.
The only isoline plotted on Fig. 1 is the one where C2 - C1 = 0; it separates the positive and negative (shaded) regions on the graph. The gradient of C2 - C1 extends at right angles to the isolines. Its orientation angle can be predicted from the ratio of C2's standard deviation to C1's standard deviation: Theta = arctan [200/233] = 40.6°, measured from a line parallel to the C1 scale and passing through the figure's origin. If the standard deviations were the same, Theta would equal 45°, and, of course, if C1's standard deviation was less than C2's, Theta would be greater than 45°.
Given this background, it can be shown that the small circle at the center of Fig. 1 has a radius of Z = 1.351, and it is tangent to the isoline C2 - C1 = 0. That is the instant when the two confidence intervals shown in Fig. 2 start to merge, and for which the JPD can be calculated. In the general case, if we know the JPD where the confidence limits of the two populations have no common values, then we can subtract this from unity to find the JPD that the two populations share.
This is the kind of problem that can be solved quickly by computer. I have written a program called TST2 NORM.EXE to find how much two normal populations overlap. The procedure requires the populations to be ordered based on the size of their means; the computer handles this once the means and standard deviations are entered.
Given the means and standard deviations of two normal distributions, the program raises Z by increments of 0.001 over the range from 0 to 4.0. For each value of Z, and for each distribution, it calculates confidence limits ñ Z units about each of the two means. The confidence limits steadily widen as Z increases. For each value of Z the program subtracts the upper limit of the distribution with the lesser mean from the lower limit of the distribution with the greater mean, and it assigns the difference to the variable "Test." In the general case, "Test" starts as a positive variable that gets smaller as Z increases.
The program searches for the value of Z for which "Test" first becomes negative; that is to say, the confidence limits of the two means begin to overlap. The last positive value of "Test" corresponds to a value of Z where the confidence ranges of the two means do not overlap, and for which the two distributions' joint-probability density can be calculated. The program then calculates the probability under the normal curve ± Z units from the mean, and it subtracts that probability from 1.0 (unity). The difference represents the joint-probability density that the two distributions share.
For the problem with the two 14C samples, TST2NORM's final screen shows the following data:
'Test'
1.071198 Z = 1.348
.7641187 Z = 1.349
.4570392 Z = 1.350
.1499597 Z = 1.351
-.1571197 Z = 1.352
There is no overlap of the range of the two distributions through Z = 1.351,
at which time the unshared Joint Probability Density reaches 0.8233.
At the interpolated value of Z where 'Test'= 0, 0.1765 of the Joint
Probability Density is shared between the two distributions.
FIRST SECOND
Mean: 12550 12965
SD: 233 200
Do you wish to run another pair? (Y/N)
Almost 18 percent of the JPD of the two populations is shared. Although we can use this as a null test (à > 0.05), in this particular case I am more interested in the actual amount of the overlap of the two populations.
I have put a self-unzipping copy of TST2NORM.EXE in the INQUA File Boutique as TST2NOMZ.EXE in case it might be useful to you. Remember that the program assumes the underlying populations really are distributed normally, and that the means and standard deviations are accurate. That is a lot to assume, but one routinely makes such assumptions with tests.
References.
Maher, L. J., Jr., 1981, Statistics for microfossil concentration measurements employing samples spiked with marker grains, Review of Palaeobotany and Palynology 32, 153-191.
Reimer, Paula, 1994, Radiocarbon calibration news. INQUA-Commission for the Study of the Holocene, Working Group on Data-Handling Methods 11, 21-23.