ALIGNS & the Nomological Network
A large language model that maps how 550,000 survey indicators relate
When researchers measure depression, anxiety or well-being, can they be sure their questionnaires capture these constructs and not something else? Psychological questionnaires decide the success of many clinical trials, yet they rely on validation principles that have barely changed since the 1950s.
In 1955, Cronbach and Meehl proposed the answer: validate measures through their place in a nomological network, “the interlocking system of laws which constitute a theory”. A construct gets its meaning from its relationships to other constructs. Seventy years later, such networks still do not exist, because a survey can only validate a handful of indicators at a time.
ALIGNS (Analysis of Latent Indicators to Generate Nomological Structures), developed with Kai R. Larsen (University of Colorado Boulder), Sen Yan, Lan Sang, Mikko Rönkkö, Ravi Starzl and Donald Edmondson (Columbia University), makes these networks operational for the first time, across more than 550,000 indicators from psychology, medicine, social policy and other fields (manuscript: ALIGNS: Unlocking nomological networks in psychological measurement through a large language model, 2025).
How it works
ALIGNS is built on the Llama 3 (8B) large language model and fine-tuned with contrastive learning: indicators of the same construct are pulled together in embedding space, indicators of different constructs are pushed apart. Training used 349,000 validated indicators from six sources, including the Semantic Scale Network, the Human Behavior Project and the Finnish Social Science Data Archive. In tests on held-out data from NIH PROMIS, information systems and organisational behaviour research, ALIGNS recognised related indicators better than leading commercial models from OpenAI and Anthropic.
Psychological constructs appear as directions in the 4,096-dimensional embedding space. Principal component analysis extracts these latent dimensions, which an LLM then names and defines from their most representative indicators.
| Nomological network | Indicators | Sources |
|---|---|---|
| Behavioral Medicine v.3 | 17,180 | UK Catalogue of Mental Health, NIH HEAL, NIH Science of Behavior Change, Stress Measurement Network; optimised against NIH PROMIS |
| Psychology | 128,456 | Top journals in eight disciplines and across all of psychology |
| Finnish–Norwegian set | 432,850 | Socio-economic indicators used by two nations to understand citizens’ needs |
Validate, visualize, explore
The system is freely available at nomologicalnetwork.org with three tools.
Validate. Researchers project their own indicators into a network. Indicators of one construct should load on the same latent dimension. Testing a 14-item “sports grit” scale, 11 items loaded on Goal Attainment, while three loaded elsewhere, for example “I put aside the other activities I enjoy and I concentrate on sport” on Group Dynamics. Embeddings, loadings and correlation matrices can be downloaded.
Visualize. The network is shown as an interactive graph: nodes are constructs, sized by their number of indicators, and edges show relationships through shared indicators. Researchers can focus on one construct and discover unexpected connections across domains.
Explore. The complete network, with variable names, indicators and their loadings on all latent dimensions, can be searched and downloaded.
Three evaluations
- Anxiety and depression. The 31 anxiety and 35 depression indicators of the NIH PROMIS instruments, among the most carefully developed in behavioural medicine, overlapped strongly (mean correlation of 0.68 across constructs) and converge into a single dimension of emotional distress. Some pairs are nearly identical in content, such as “I felt indecisive” (anxiety) and “I had trouble making decisions” (depression), r = 0.87.
- Child temperament. Two independent experts classified 310 of the 383 indicators on the child temperament dimension as temperament (81% precision). They suggested four potential new dimensions (social engagement, child worry, emotional regulation, compliance) and questioned an existing one, sensory sensitivity.
- Applicability check. Seven psychometric researchers, from a doctoral student to professors with more than 20,000 citations, found the system valuable for exploration and for re-evaluating their own scales. One senior researcher discovered that two of his own scales, published over a decade earlier, loaded on a single “super-construct”.
Health measurement and next steps
A PROMIS-fine-tuned version of the model decomposed 17,180 behavioural medicine indicators and helped validate and clarify the NIH PROMIS scales (Advances in Patient-Reported Outcomes, 2025). With Columbia University and CU Boulder, I co-authored a successful NIH proposal on stress ontologies and AI-assisted tools for ontology development (2023–2024).
Next questions:
- Reliability of LLMs as measurement instruments, using generalisability theory to decompose variance into model, prompt and sampling facets.
- Hierarchical construct structures through formal concept analysis.
- Valid downstream inference when LLM-derived scores replace survey measures.
ALIGNS complements the ITEM Ontology, which describes the inner structure of each indicator explicitly. Results of this research line are incorporated in PsyProxy.