William J. Phalen

Institute for Job & Occupational Analysis

Robert M. Yadrick

Technical Training Research Division

Armstrong Laboratory Human Resources Directorate

Walter G. Albert

Manpower and Personnel Research Division

Armstrong Laboratory Human Resources Directorate

The next study by Yadrick, et al. (1994) described the application of a new, univariate procedure for clustering the weighted expressions into groups of equivalent (or synonymous) expressions. It appeared to the researchers that the expressions within each group were sufficiently synonymous with one another that a single expression might be picked from each group to represent an equal interval scale with the optimal number of points. It remained to be determined whether the divisions or cut points suggested by the groups in the cluster solution had captured the "psychologically real" levels of a frequency or amount scale embedded in the ordered lists of magnitude-estimated expressions. Therefore, a validation study was undertaken to determine the degree of correspondence between mathematically defined cluster groups and perceptually defined, or psychologically real, groups of expressions. This paper will report on the development and application of the criterion measure used to validate the univariate clustering procedure.

In this study, the equivalence equation was applied first to determine the equivalence of the top two expressions in the list, i.e., expressions "1" and "2" on the lefthand side of Table 1 for "frequency" and likewise on the lefthand side of Table 2 for "amount". If the equivalence value exceeded 80.00%, then the equivalence of expressions "1" and "3," then "1" and "4," etc. was computed, with expression "1" repeatedly being used as the target expression, until the equivalence value fell below 80.00%. At this point, the set of expressions falling within a minimum linkage of 80.00% was selected as the first group of expressions representing the highest level of the scale. Thus, in Table 1 (lefthand side) expressions "1" through "3" formed the first group of equivalent frequency expressions with a minimum equivalence (between expressions "1" and "3") of 83.81%. The next target was expression "4." Expressions "4" and "5" were compared, "4" and "6," etc. until "4" and "10" yielded the lowest acceptable equivalence of 80.66%. The remaining groups were formed in a similar manner, resulting in 12 "frequency" groups (or scale levels) and 13 "amount" groups (or scale levels). The minimum equivalence values are reported to the right of each group.

Column 1 of Tables 1 and 2 also shows the magnitude-estimated weights for each expression, as derived by Bass, et. al. Although the groups of expressions generated by this clustering procedure seemed to be very reasonable in the judgment of the authors, it could not be assumed that the groupings represented psychologically real divisions without comparing them against a criterion of psychologically derived groups of expressions.

A sample of 42 respondents, consisting of behavioral scientists and clerical workers at the Armstrong Laboratory and at three contractor offices provided responses. Approximately equal numbers of each version of the survey were completed to provide the desired counterbalancing.

The groups identified by the 42 respondents were consolidated into a matrix whose rows and columns indicated the number of times each expression was selected as a "beginning" or "ending" expression, respectively. Evaluation of the row and column totals made it possible to select the set of groups which provided the best overall fit of the individual respondent data. The resultant sets of psychologically defined groups based on the perceptual judgments of 42 respondents are shown in the righthand portion of Table 1 for the "frequency" expressions and in the righthand portion of Table 2 for the "amount" expressions. The minimum equivalence values for the psychologically defined groups are also reported as additional points of comparison with the mathematically defined groups.

It might be argued that the expressions constituting the singleton groups should be dropped, especially "seldom," which is clearly out of place. This would reduce the number of mathematically defined groups to nine, while the psychologically defined groups would remain at eight. This would also increase the correspondence between the two clusterings. A major point of difference involves expressions "28" through "35" (if "seldom" is eliminated). The mathematical clustering separated this set of expressions into three groups, while the psychological clustering considered the set to be one group. In this case, the psychological grouping makes more sense. However, it should be noted that the minimum equivalence for this group (expressions "28" through "35") is only 0.19, which is to say that there is an extremely large ratio difference between a weight of 4.72 for "very seldom" and a weight of .33 for "seldom." If "seldom" is dropped, the minimum equivalence for the group would jump to 48.84, which is still low. Overall, the correspondence between the mathematical and the psychological clustering is reasonably good, especially considering some of the questionable weighting of expressions.

In the mathematical clustering, the singleton group containing "a limited amount" should be dropped. It is another one of those fuzzy expressions that is hard to define clearly, since all amounts other than "all" are "limited" amounts. Where the mathematical and psychological groups do not correspond, it would appear that the mathematical grouping at the upper end of the scaled list is superior, but the psychological grouping at the lower end is superior.

Albert, W.G., Phalen, W.J., Selander, D.M., Dittmar, M.J., Tucker, D.L., Hand, D.K., Weissmuller, J.J. & Rouse, I.F.
(1994, October). Large-scale laboratory test of occupational survey software and scaling procedures. In the
symposium, Bennett, W. Jr., Chair, Training needs assessment and occupational measurement: Advances from
recent research. *Proceedings of the 36th Annual Conference of the International Military Testing Association. *
Rotterdam, The Netherlands: European Members of the IMTA.

Bass, B.M., Cascio, W.F., & O'Connor, E.J. (1974). Magnitude estimations of expressions of frequency and amount.
*Journal of Applied Psychology***, 59, **313-320**.**

**
**Yadrick, R.M., Phalen, W.J, Albert, W.A., Dittmar, M.J., Weissmuller, J.J., & Hand, D.K. (1994, October). Clustering
of magnitude estimations of frequency and amount of time. *Proceedings of the 36th Annual Conference of the
International Military Testing Association*. Rotterdam, The Netherlands: European Members of the IMTA.

Yadrick, R.M., Phalen, W. J., Albert, W.G., Dittmar, M.J., Weissmuller, J.J., & Hand, D.K. (1993, November). Magnitude estimations of frequency and amount of time. Paper represented at the 35th Annual Conference of the International Military Testing Association. Williamsburg, VA: U. S. Coast Guard.