Speaker
Description
Classification is the activity of assigning objects to some pre-existing exclusive categories that form a comprehensive spectrum (scale) of the studied property. The classifier can be a person, a machine, an algorithm, etc. Classification accuracy is a combination of trueness and precision. The latter (precision), perceived as the 'closeness of agreement' between results of multiple classifications of identical items under stipulated conditions, is split into two components. The intra-component (repeatability) characterizes the dispersion of results from the same classifier. The inter-component characterizes the dispersion of results due to the participation of different classifiers in the classification process (e.g., in collaboration studies).
Identity is the key concept that distinguishes the metrological perception of "closeness of agreement" from the one accepted in the field of social sciences (such as Fleiss' kappa measure, for example). Identity is necessary to isolate that part of the non-reproducibility that is associated only with the measurement procedure. Two distinctive features of the proposed measure of expected agreement between independent classifiers are:
- It obeys the superposition principle, i.e. the total measure equals the weighted sum of partial measures, where the ‘‘weight’’ of every category is the probability of an item in the test to belong to this category.
- For any number of classifiers, it is expressed through repeatability and reproducibility variations only.
We intend to demonstrate the proposed measure using various practical examples.
| Classification | Both methodology and application |
|---|---|
| Keywords | classification, agreement, intra and inter variations |