What Is Psychometrics?
Psychometrics is the scientific field concerned with the measurement of psychological attributes — personality traits, cognitive abilities, attitudes, and other mental characteristics. It provides the technical standards that separate rigorous psychological assessments from the pop-psychology quizzes that flood the internet.
The core question psychometrics addresses: how do you measure something that can't be directly observed? You can't weigh Conscientiousness or put Extraversion in a graduated cylinder. Psychometric theory provides the methods for creating, validating, and interpreting indirect measurements through behavioral responses — what we call "test items."
The Foundation: Reliability
Before a test can be valid, it must be reliable — it must produce consistent measurements. Four types of reliability matter:
Internal Consistency
Do the items within a scale correlate with each other? If you have a 10-item Extraversion scale, do people who endorse one item tend to endorse others? This is measured by Cronbach's alpha (α), ranging from 0 to 1. The standard benchmark:
- α ≥ 0.70: acceptable
- α ≥ 0.80: good
- α ≥ 0.90: excellent (sometimes too high — suggests redundant items)
Test-Retest Reliability
Does a person get the same score if they take the test again later? For stable trait measures, we want high test-retest reliability over short intervals (weeks to months). Big Five measures show test-retest correlations of 0.75-0.85 over periods of weeks. Over years, reliability decreases — reflecting genuine personality change, not measurement error.
Inter-Rater Reliability
For observational measures, do different raters give the same ratings? Critical for clinical assessment, behavioral coding, and performance evaluations.
Parallel Forms Reliability
Do different versions of the same test (alternate forms) produce equivalent scores? Used when test forms need to be varied to prevent practice effects.
Validity: What Does the Test Actually Measure?
A test can be highly reliable without being valid — it consistently measures something, just not what it claims to measure. The three pillars of validity:
Construct Validity
Does the test measure the theoretical construct it's supposed to measure? Established by showing:
- Convergent validity: The test correlates with other measures of the same construct
- Discriminant validity: The test does NOT correlate with measures of different constructs
- Nomological network: The pattern of correlations matches what theory predicts
For example, an Extraversion scale should correlate with other Extraversion measures (convergent), should not correlate strongly with Conscientiousness (discriminant), and should predict social behavior, positive affect, and leadership emergence (nomological).
Criterion Validity
Does the test predict real-world outcomes?
- Concurrent validity: Scores correlate with current criteria (test scores correlate with current supervisor ratings)
- Predictive validity: Scores predict future outcomes (personality test at hiring predicts job performance 2 years later)
Predictive validity is the gold standard for applied assessments. Barrick and Mount's (1991) meta-analysis found Conscientiousness (ρ ≈ 0.22) and Emotional Stability (ρ ≈ 0.19) predict job performance across occupations — establishing criterion validity for the Big Five in employment contexts.
Content Validity
Do the items adequately sample the full domain of the construct? A test measuring "career satisfaction" that only asks about salary misses the broader construct. Content validity is typically established through expert review and systematic domain mapping before item writing.
Factor Analysis: How Personality Structure Is Discovered
Factor analysis is the primary statistical technique used to identify personality structure. The logic: if there's an underlying trait called "Extraversion," then items measuring sociability, assertiveness, positive affect, and excitement-seeking should all correlate with each other — because they're all expressions of the same underlying tendency.
Factor analysis identifies these correlation clusters mathematically, revealing the latent structure beneath a set of variables. The Big Five personality framework emerged from factor analyses of two sources:
- Lexical analyses: Factor analysis of all personality-descriptive adjectives in the English language
- Questionnaire analyses: Factor analysis of existing personality scale items
Both approaches consistently recovered five robust factors — Openness, Conscientiousness, Extraversion, Agreeableness, and Neuroticism — providing remarkable convergent evidence for the Big Five structure.
What Separates Rigorous Tests from Poor Ones
The MBTI Problem
The original Myers-Briggs Type Indicator has several well-documented psychometric limitations:
- Forced dichotomies: Forcing continuous traits (Introversion-Extraversion) into binary categories loses information and produces unstable classifications
- Test-retest instability: 40-75% of people test as a different "type" when retested 5 weeks later
- Limited criterion validity: MBTI types do not predict job performance comparably to Big Five traits
- Ipsative scoring: Some versions use forced-choice formats that prevent comparison between individuals
This doesn't mean MBTI is useless — it has value as a reflective framework. But it should be interpreted as an approximation, not a precise measurement.
Internet Personality Tests
Most internet personality tests lack published reliability data, factor-analytic support, or validation studies. Red flags:
- No published Cronbach's alpha values
- No test-retest reliability data
- No peer-reviewed validation studies
- Type results without continuous score information
- Proprietary scales with no independent validation
How JobCannon Approaches Test Quality
JobCannon's assessments are built on validated frameworks with published psychometric evidence. The Big Five uses a well-validated item structure with documented reliability. The Psychometric Assessment measures cognitive aptitude through items with known difficulty and discrimination parameters. The Knowledge Base documents the scientific foundation behind each test methodology — because transparency about what is and isn't measured is part of what makes assessment trustworthy.