The ITEM Ontology
The semantic anatomy of psychometric indicators
Survey research rests on its indicators, the individual questions that measure constructs such as stress, trust or ease of use. Information Systems has rigorous statistical procedures to validate scales. Yet statistics cannot reliably detect semantic, lexical and logical problems in the wording of indicators: scales can show good psychometric properties even when their items are vague, double-barreled or, in extreme cases, meaningless.
The Indicator Terminology for Explanation and Measurement (ITEM) Ontology provides a shared language to take indicators apart. Instead of treating an indicator as an irreducible whole, ITEM decomposes it into its entities and the relationships between them, so that its meaning can be analysed, discussed, peer-reviewed and corroborated.
The work, led by Kai R. Larsen (University of Colorado Boulder) with an international author team and supported by the U.S. National Institutes of Health (NIH), appeared in Information Systems Research (2026, VHB A+, ABDC A*).
Four building blocks
ITEM builds on DOLCE, a top-level ontology for linguistic and cognitive engineering, and on Toulmin’s argumentation structure. It comprises 42 entities in four high-level hierarchies, nearly always present in a single indicator, plus 24 relationship types:
- Objects: physical and non-physical objects, agents such as the respondent, and the information system.
- Measurables: what an indicator actually measures: an activity, an attribute (including mental attributes), a value, or a cause/enable relationship.
- Qualifiers: 13 qualifiers that set the frame of reference, for example temporal, spatial, conditional, hypothetical or cognitive (“I find the system useful” versus “The system is useful”).
- Response sets: nominal, ordinal and continuous response formats, such as agreement or frequency scales.
The figure at the top of this page shows the full hierarchy and which parts were adopted or adapted from DOLCE and Toulmin, and which were newly added.
Taking an indicator apart
Consider the indicator “I feel busy due to use of ICTs at home”, answered on a strongly-disagree-to-strongly-agree scale:
Entities of an example indicator coded with ITEM (colours as in the ITEM codebook).
Relationships then connect the entities: busy is an attribute of the respondent, at home qualifies the activity use, and use causes busy. The key question is which entity the response set actually measures, the target measurable. Here, respondents are not asked how busy they feel or how much they use ICTs, but whether use causes busyness. The target measurable is therefore the Cause/Enable relationship. Splitting the indicator in two would measure busyness and use separately and leave their relationship to empirical analysis.
Finding problems in indicators
A literature review identified 16 indicator quality concerns in six areas: temporal context, vagueness, alternative reality, complexity, response set effects and construct grouping. ITEM gives each concern, for the first time, an operational definition as a pattern of entities and relationships.
Take double-barreled indicators, which the literature variously describes as containing several “ideas”, “objects”, “questions” or “things”. In ITEM terms, an indicator is double-barreled when a response set points to two or more target measurables. In “I feel alive and vital”, a respondent who answers “not true at all” may reject one attribute or both, and we cannot tell which.
The Python tool ITEMIZER applies these operational definitions to coded indicators and flags potential concerns. Applied to three instruments from top journals, it found concerns in all of them; the most frequent was measuring activities without specifying a time frame. ITEM also enables a new, entity-level form of content validation, which checks whether indicators over- or under-represent their construct definition.
Evaluation
- Reliability: independent coders reached high agreement (Bennett scores of 0.80 to 0.95 for role identification; Cohen’s kappa of 0.66 to 0.84 for role classification).
- Representativeness: ITEM fully represented the indicators of three IS theories and 99 randomly selected IS indicators.
- Applicability check with 11 senior IS researchers: after a 45-minute training, they rated the indicators of a published construct as significantly less clearly worded and less content-valid than before (p < 0.01), and rated ITEM as highly useful.
- External validity: a workshop with survey and polling experts at Gallup found ITEM important and suitable beyond IS. One participant: “I LOVE that list of qualifiers.”
The ITEM toolset
At itemontology.org, researchers can watch tutorial videos on each entity type, code their own indicators in the BRAT annotation tool configured with ITEM, and generate an ITEMIZER report of potential problems. The ontology itself is available in OWL format on the Protégé ontology portal.
Why it matters
ITEM shifts the view of indicators from a purely statistical to a linguistic and ontological perspective. It revealed, for example, that IS research uses two basic kinds of indicators, those that measure an entity directly and those that measure a relationship between entities, a distinction not made in prior work. And it points to a new form of validation, ontological validation, in which ITEM mediates between a construct definition and its measurement items.
ITEM complements ALIGNS: ALIGNS locates items in a learned semantic space, while ITEM describes their structure explicitly. For larger IS-specific vocabularies, ITEM can be extended with the Information Systems Ontology.