William E. Alley
Melody M. Darby
USAF Armstrong Laboratory
Human Resources Directorate
Manpower and Personnel Research Division
7909 Lindbergh Drive
Brooks AFB, TX 78235-5352

Cheng Cheng
Johns Hopkins University

A Monte Carlo simulation study was conducted to estimate the proportion of successful employees expected as a result of optimal personnel selection and classification. Tables originally prepared by Taylor and Russell (1939) for evaluating a single-job context were extended for use with multiple jobs. Expected success rates of assigned personnel that varied in the original study as a function of a) the base rate for success under random selection, b) the selection ratio and c) the validity of the test composite, were also shown to differ according to the number of potential job categories available for assignment. Applications of the table for more effective human resource management were discussed.

In this paper, we address the problem of how validity coefficients affect the practical benefits of tests in selection and assignment. Taylor and Russell (1939) considered the problem of selection for a single job category. They showed that the proportion of successful employees after selection is a function of the selection ratio -- the proportion of applicants selected from those who apply, the base rate -- the percentage of employees who would be successful under random selection procedures, and the validity of a proposed selection test. We extended their work to situations where selection and assignment occur simultaneously for multiple job categories. The military services have unique requirements for filling large numbers of vacancies in different occupational codes from a common applicant pool. Brogden (1959) addressed a similar problem but used a different measure of employee performance -- the mean performance score of those selected. Recent work to extend the Brogden tables beyond 10 jobs (Alley & Darby, 1995; Cheng & Darby, 1996) led us to consider the problem initially posed by Taylor and Russell in the context of classification decisions. The purpose of the present study was to explore the theoretical benefits of having more than one alternative assignment on the metric of "proportion of successful employees." We demonstrate how the expected benefits change when the number of assignment opportunities in a personnel selection and classification system increases from one to 10 jobs.

Elements from the mean performance calculation problem for multiple jobs addressed by Brogden (1959) were applied to extend the single-job Taylor-Russell results. Distribution theory from mathematical statistics, Monte Carlo simulation techniques, and probability and quantile (percentile) functions from currently available statistical analysis packages were used to calculate the results. The following terminology was used and assumptions made. The Criterion, , is the measure of satisfactory employee performance and q is the proportion of employees performing satisfactorily. It is assumed that the employee population and an applicant population are similarly constituted regarding the potential for satisfactory performance. A Selection Battery, , is administered to the applicant population. Applicants are divided into two groups, accepted or rejected for employment, on the basis of a selected critical score on this test. The proportion of rejected applicants is the rejection rate,. The selection ratio is the proportion of selected employees,. Successful performance in the multiple job case is defined as successful performance in their assigned job from among the full set of jobs. Additionally, as in Brogden (1959), we make the simplifying assumptions of an infinite pool of jobs and applicants and an intercorrelation among jobs of zero.

The problem is formulated as follows. Suppose there are jobs and applicants. Each applicant can be represented by a vector of m test composites (random variables) from a Selection Battery,. Scores are distributed as independent standard normal deviates. An applicant is assigned to a job according to his highest test score, i.e.,

is then the largest order statistic of a series of random variables. The distributional properties, mean, variance, and percentiles, of order statistics are well known (Hogg and Craig, 1970). The distribution of the criterion, , is related to the selection criterion, , by the following model:

where is distributed as above and, a random perturbation, is distributed as a standard normal deviate. and are related by the validity coefficient,. The model constants, and, can be resolved in terms of allowing an opportunity to generate values of from values of as in the following argument.

where

Then

When then.

When then.

When We can set:.

Now, with and specified, the proportion of successful employees from among a selected group can be expressed as the conditional probability that exceeds a percentile, , given that exceeds a percentile,. For purposes of calculation the conditional probability is evaluated using the relationship:

For given values of, this probability was evaluated using Monte Carlo techniques. The moments of were evaluated numerically with a simple trapezoidal rule. Percentiles for, , were calculated as established in Cheng and Darby (1996) with functions in the SAS (1990) statistical analysis package. Values of were generated from using Monte Carlo simulation techniques and percentiles of were calculated using simple descriptive statistics, the sample percentiles of the simulated values of. Sample size for the simulations was N = 100,000.

As a preliminary check on the computational procedures, the simulated values for a single job were compared to those in the original Taylor-Russell table. These comparisons showed a correspondence of +1% among the estimates from both sources. This was considered sufficient accuracy for purposes for which the table will be used.

Table 1 shows the results of the analyses for representative combinations of assignment system parameters and number of jobs ranging from 1 through 10. A graphic depiction of that portion of the table corresponding to a base rate of .40 and a selection rate of .60 is plotted in Fig. 1. As reflected in the original Taylor and Russell table, the proportion of successful employees assigned to a single job increases from 42% when

validity is .10 to 65% when validity is .90. The effect of increasing validity on the proportion of successful employees is approximately linear throughout the effective range of the test. When there are two possible opportunities for job assignment (assuming the correlation between jobs r = 0), the overall employee success rate increases from 43% to 89% over the same range of validities. With 10 jobs, the corresponding values for validities ranging from .10 to .90 were 45% to 100% of employees considered successful.

The functional form of the values shown in the figure is characteristic of all those we calculated as validity and number of jobs increases. At fixed levels of validity, the percentage of selected and assigned employees considered successful increases as a negatively accelerating function in the lower range of jobs (1- 5) and then levels off to asymptotic values as the number of jobs increase. With number of jobs fixed, percent successful increases in more or less linear fashion when there are relatively few job opportunities. With larger numbers of jobs, the function again shows negative acceleration for lower values of validity, reaching asymptotic levels as validity approaches 1.0. Thus, the effects of validity on the success rate are approximately linear when there are few job opportunities -- one to 5. However, in the range of 6 to 10 jobs, the effects of increasing validity are most apparent in the range .10 to .50. As validities increase toward unity, there is a positive but diminishing effect on employee success rate.

Human resource managers are often called upon to make investment decisions that could benefit their organization in terms of maintaining or improving the quality the workforce. Alternatives present themselves in the form of questions that imply different costs, benefits and trade-offs for alternative courses of action. For example, would an incremental investment in recruiting that might result in a more favorable selection ratio produce a gain in employee success rate comparable to an alternative investment to improve the validity of a selection test? If the current R = .50 for 4 job categories, and the base rate is 40% based on selecting 60% of the applicants, one would need to increase the validity of the test battery by 10 points to R = .60 in order to obtain benefits equivalent to extending the applicant pool so that the selection ratio decreases by 20 percentage points.

A major limitation to the practical use of the table is that estimated gains in the number of successful employees will be attenuated to the extent that the job performance estimates among jobs are correlated. That is, if estimated success for applicants in Job A is related statistically to estimated success in Job B, then the benefits of two job opportunities versus one will be less than if the estimates were statistically independent. In the most extreme case, where job requirements totally overlap, the advantages of the second job category vanish entirely. Further work is underway to develop adjustment procedures for applications where correlations of performance estimates between jobs exceed 0.

Alley, W.E., & Darby, M.M. (1995). Estimating the benefits of personnel selection and classification: An extension of the Brogden table. Educational and Psychological Measurement, 55(6), 938-958.

Brogden, H.E. (1959). Efficiency of classification as a function of number of jobs, percent rejected, and the validity and intercorrelation of the job performance estimates. Educational and Psychological Measurement, 19, 181-190.

Cheng, C., & Darby, M.M. (1996). A Revision of the Brogden Table. Unpublished manuscript. Brooks AFB, TX: Armstrong Laboratory, Human Resources Directorate.

Hogg, R.V., & Craig, A.T. (1970). Introduction to Mathematical Statistics (3rd ed.) New York: Macmillan Publishing Co.

SAS Institute Inc. (1990). SAS Language: Reference, Version 6, 1st Ed. Cary, NC: SAS Institute, Inc.

Taylor, H.C., & Russell. J.T. (1939). The relationship of validity coefficients to the practical effectiveness of tests in selection. Journal of Applied Psychology, 23, 565-578.

Back to Table of Contents