HP3802

Foundations of selection

• Selection = process of collecting + evaluating information about individual to extend offer of employment; basis of employee performance (need maximum performance to develop competitive advantage in global competition)
• Work performance = ability (related to selection + training) + effort (related to motivation)
• Collecting and evaluating information - systematically collected by selection specialist
• WRC = attributes related to performance (including personality)
• KSA = factual information of how to do + degree to which individual can perform activities (excluding personality)
• Selection for initial job and promotion
• Initial job: many applicants, external to organisation (commonly graduates etc), recruited via formal mechanisms, everyone has equal chance to demonstrate KSA; selection instrument used to reduce applicants to favourable selection ratio
• Promotion: internal candidates, limited formal recruitment techniques (can rely on existing information eg past performance); leads to Peter Principle (advancement in position based on performance in previous role)
• Issues: lack of job-related data, inconsistent information, non-formalised decision rules
• Future interests
• Applicant market conditions
• Administrative rulings eg EEO
• Match of (future) interests between employee x organisation -- maintains effects over time

Selection and strategic HRM

• Strategic HRM = used to align number of employees + performance with goals of firm; requires HR system to be coordinated + interact smoothly with one another
• Addressing HR issues: forecast future demand for labour + when should these people be employed + how many individuals will be present in each job with current HR programme system -> compare results of demand forecast vs labour supply forecast
• Interactions of selection with strategic HRM system: selection <-> recruitment (both deal with individuals not working for the team, need to attract + increase probability of accepting when offered) // <-> initial training (affected by level of KSA of individuals selected) // <-> compensation /<-> job performance measurement; all systems contribute to maximum benefit (resource-based theory: organisations gain advantage over competitors via valuable resource in short supply)
• Outcomes of HRM
• Profits
• Productivity + quality
• Turnover

Developing a selection program

• Effective selection program: collect information from applicants closely related to job performance + effectively use information to identify best applicants
• Steps
• Job analysis information: gathering information about job (eg tasks, results, material, environment) --> convey information to potential applicants + as database for other steps in development process
• Identify job performance measures - may obtain information from supervisor judgement / other team members
• Identify worker characteristics: KSA + other characteristics necessary for successful perfromance
• Assessment devices - must measure specified KSA + differentiate among applicants
• Validation: empirical (scores from representative samples + measures of performance in important parts) + content (systematically take data -> determine relationship between test x performance)
• Constraints
• Limited information on applicants (often limited by cost of obtaining data)
• Cross-purposes of applicant vs organisation
• Measurement of jobs / people / work performance - need to ensure accuracy of descriptions
• Factors affecting work performance, eg training / feedback / goal-setting / compensation
• Research vs practice: practice managers may not have research / content knowledge
• Selection vs staffing: selection as essential foundation for effective staffing (involve broader strategic HRM system)

Trends in selection

• Big data: increasingly growing, complex testing + gathering of data
• Magnitude of demographic group differences + effects on work-related characteristics
• Internet-based selection measures

I/O psychologists role in selection

• Identify KSA necessary for performance
• Develop + identify measures
• Research on relationship between measures x performance (may need to go into organisation to see what is working vs what is not)
• Evaluate evidence of fair treatment
• Enhance current methods to improve prediction

Recruitment

• Recruitment = organisational activities that influence number + types of individuals who apply for a position, also affect applicants decisions about whether to accept a job offer
• Recruitment vs selection objectives: present positive expectations of organisation (organisational branding) vs having applicants to know what organisation is truly like // present positive information about self vs need for organisations to have accurate information
• Purposes of recruitment
• Appropriate number of minimally acceptable applicants (must demonstrate value, have adequate qualifications) +\ keeping costs reasonable
• Organisation's legal and social obligations of demographic composition -- applicant pool must be demographically balanced
• Reduce percentage of applicants poorly qualified / wrong KSA; related to cost of selection / wastage

Planning and executing recruitment

• Process of recruitment (together with screening + self selection): potential applicant population -> applicant pool (chosen to apply \ may not have accepted) -> job candidates -> hire
• Steps
• Objectives: how many individuals? What KSA? Demographic diversity? When?
• Strategy: which recruitment activities? When? How? By whom? What message?
• Activities
• Recruitment sources (affected by cost of use / amount of information presented / predictability of number of applicants; different effects due to realistic information hypothesis / individual difference hypothesis; need to also consider nature of job / image of company in labour market / attitudes of current employees / demographics of applicants)
• Recruitment personnel (recruiters as first real people applicants encounter after they get through website \ weaker influence than members of intended work group)
• Content of message (RJP most powerful effect @ applicants with no opinion about job / with relevant experience with job attributes / with self-insight; decisions influenced by perception of firm's image + familiarity with firm)
• Administration of recruitment (especially promptness of follow up)
• Results: performance appraisal rating for new employee + % voluntary leave + % manager satisfaction + performance on training measures eg performance tests / cost per hire (internal costs / external costs / company visit / direct fees) + recruitment website visitation patterns

Managing applicants during recruitment

• Attracting / generating interest (must balance between sell vs realism)
• Establish recruitment objectives -> make strategic decisions: type of applicants? Broad vs person-to-person? Message orientation (positive vs realistic, how much)?
• Predictors of attraction: organisational reputation / size / age / profitability / life cycle of firm / product awareness / location / social relationships
• Persuasion: does applicant pay attention? Understand message? Relevant behaviour change?
• Targeted recruiting: high cognitive ability (job as challenging / stimulating etc) / high conscientiousness (reliable / well-organised etc) / racial minorities (innovative / progressive etc)
• Effectiveness of recruiting methods: media richness + credibility (expertise + trustworthiness) + degree of attention
• Consider referrals / walk-ins / more informal methods
• Maintaining interest - recruiter must be effective administrator, use two-way communication, strategic thinking about information
• Applicant self-selection - organisation must present accurate fit information, fairnessbased on perceptions of procedural (consistency) / informational (timeliness of feedback) / interpersonal justice
• Recruiter must be seen as competent / informed / trustworthy / warm etc
• On-site visit communicates more comprehensive information
• Post-offer closure - applicant decision influenced by compensation / organisational culture / work
• Applicant tracking system / human resource information system promotes prompt responses; tracks through each stage
• Global recruiting - cross-cultural differences in socio-economic conditions, also depends whether white vs blue collar

Models of job performance

• Campbell's model (lacks opportunity)
• Ability / personality / interests / education / experience -> declarative knowledge // procedural knowledge (core task proficiency / personal discipline / effort / non-core proficiency / written and oral communication / facilitating peer and team performance / supervision and leadership / management and administration; first three are universal)
• Declarative knowledge * procedural knowledge * motivation -> job performance
• Tacit knowledge (related to procedural knowledge) = skills / ideas / experiences difficult to transfer by means of writing or verbalising, eg hard-to-pin-down skills / mental models / ways of approaching problems / organisational routines
• Crossover of CWB and OCB
• Blumberg & Pringle's model
• Performance = capacity (physical / cognitive / KSA / education) * opportunity (environment / job design / group dynamics) * willingness (personality / motivation / values)
• Ways to improve opportunity to perform: technology / more efficient workflow systems / delegation of challenging and important tasks

Job performance measurement

• Performance measurement = performance assessment measures to enhance individual performance + organisational performance to achieve business goals
• Purpose of job performance measures: serve as criteria measures in validation studies
• Model of performance: performance (actions / behaviours; easier to measure than effectiveness) -> effectiveness (evaluation of results) -> productivity (output :/: input)
• Work characteristics measured in addition to task behaviours
• Different management disciplines, eg strategic management / process improvement / performance appraisal
• Indicators of performance: organisational (financial / market; more related to strategic management systems) / individual (more related to HR initiatives eg selection)
• Key questions: who's rating? How competent is the rater? Requirements of organisation? Realistically expect someone to fulfil requirements? Can tangibly measure?

Types of job performance data

• Production data: quantity / quality of output as direct result of job actions; usually based on specific nature of job tasks \ cannot gather information on individual data
• HR personnel - countable data; almost always @ individual basis
• Absenteeism
• Turnover
• Grievances
• Accidents (injury / damage to equipment)
• Promotions (assumption that job performance x-> promotion)
• Training proficiency: measure of employee performance immediately following completion of training
• Judgements made by instructor about trainees
• Pen-and-paper
• Work sample
• Judgemental data - work judged by individual familiar with it eg direct supervisor
• Trait rating scales (*no relationship to performance)
• Simple behavioural scale: rate on critical tasks of job
• BARS (performed work behaviour) / BES (expected work behaviour) - defined rating points in relation to job behaviour
• CARS: computerised, rater select between two statements on a specific performance dimension -> two more statements

Problems with judgemental measures

• (Inadvertent) bias / rater error: halo or horns (may rate equally on different performance scales) / leniency (disproportionate number of workers in high ratings) / severity (opposite of leniency) / central tendency -- require training of supervisors, eg rater-error training (videotaped interactions -> own rating of person vs manager's rating -> discussion + solutions) / stressing rater accuracy (pay close attention to employee performance in relation to dimensions; group discusses types of behaviour indicative of various levels within each dimension + associated performance)
• Unsystematic data collection - incompleteness of data forces judgements on partial impressions eg primacy / recency
• Psychometric characteristics of judgemental data
• Judgement x production measures; production measures tend to be narrower
• Reliability / validity of performance ratings - performance change over time? Situational variables? Idiosyncratic rating tendencies? Organisational perspective relevance to person being rated?
• 360-degree feedback - can ratings be averaged? Individual biases? Explanations given to raters about how rating is used (development vs compensation / promotion)?
• Implications: behaviour rather than trait scales, dimensions being broad statements of performance, extensive training done with supervisors using instrument

Characteristics of selection criteria measures

• Individualisation: must measure distinct performance of individual
• Controllability: must be within employee's control
• Relevance
• Measurability: must generate a number \ what can really measure effectiveness?
• Reliability - what if there are fluctuations?
• Variance - can be affected by standardisation in output / inappropriate use of measurement device
• Practicality: cost and logistics
• Lack of contamination - particularly a problem in AC (data used to make decisions about whom among the group should be promoted)
• Specificity: use job performance data specifically collected for selection rather than for other HRM purposes

Criterion use

• Multiple scores vs composite: straightforwardness (better used for research) \ reality of multiple tasks within jobs (better used for managerial decision making)
• Combining measures into one
• Dollar criterion: develop composite job performance measure that represents economic worth; part of utility analysis
• Factor analysis: based on intercorrelation matrix -> statistically combine separate measures into clusters (each can serve as composite performance measure), also provides weights to be applied to specific measures; may be subjective
• Expert judgement - can use preferences from individuals (eg customers) who derive value from organisation's outputs for weighting of performance criteria

Practical problems with criterion

• Reliability: consistency of criterion; can be subjected to external factors including untrianed supervisors -- use more objective performance outcomes, prefer BARS, provide training, consistent application of appraisal system
• Validity: may not serve original purpose; often related to hard criteria -- base performance appraisals on thorough analysis of job requirements, avoid indicators unrelated to individual qualities
• Range restriction (can be problem for both predictor and criterion) -- ensure validation sample is representative of all levels of test / job performance
• Criterion deficiency / contamination
• Relevance (overlap between theoretical x actual)

Organisation citizenship behaviours

• OCB = done by worker to assist other workers to organisation (eg teaching new workers, assisting other workers, putting extra time and effort into work); not formally part of job \ may be used as precursor to performance
• Dimensions of OCB: helping / sportsmanship / loyalty / compliance / initiative / civic virtue / self-development
• Linked to organisational commitment, perceptions of fairness, leader supportiveness
• Managers may be influenced by OCB especially in judgemental performance evaluations
• Measured using self-report judgemental scales

Adaptive performance

• Adaptive performance = deliberate change in thinking / behaviour because of anticipated / existing change in work activities
• Difference in work-related characteristics can be used to predict differences in AP, eg cognitive complexity (integrate conflicting information) / frame changing (alternate between multiple ways of problem solving) / resiliency (persist and recover quickly) / problem solving (persist and work through details of problem) / learning ability (apply lessons learned from previous experience)
• Dimensions of AP (cannot be observed directly): handling emergencies / handling work stress / creative problem solving / dealing with uncertain work situations / learning work tasks and procedures / interpersonal adaptability / cultural adaptability / physical oriented adaptability
• VUCA: volatile / unpredictable / complex / ambiguous

Counterproductive work behaviour

• CWB = undesirable performance actions (intentional) that harm organisation (often also employees + customers)
• Types of CWB: individual deviance / organisational deviance
• Moderately negatively correlated with OCB \ not single continuum of behaviour
• Integrity tests developed to identify applicants with high probability of CWB

Role of job analysis

• Job analysis = purposeful, systematic process of collecting information on important work-related aspects; give an idea of what to predict to test for validity
• Work activities: what / how / why / when
• Tools / equipment
• Context of work environment (eg work schedule / physical work conditions)
• Requirements eg KSA
• Use of data
• Identify necessary KSA for success
• Select / develop selection procedures for important KSA --> administered to job applicants + forecast likely successful employees
• Develop standards of performance for success
• Framework for application to selection: job analysis method -> task-related measures --> human attributes (work-related characteristics, usually inferred by expert judgement) --> selection procedure; cross-validate selection procedure <--> performance measures (<- work-related measures)

Issues in job analysis

• Growth in job analysis - developed because nature of job changes with technology / seasons
• Legal issues -- job analysis is mandatory, must be for job for which selection procedures are used, must be in writing, describe detail, collected from variety of current sources, sample size of individuals as SMEs should be large and representative, most important tasks represented, competency levels for entry-level jobs specified
• Collecting job information
• Employee specification for changing jobs / jobs yet to be created

Job analysis methods

• Task-oriented vs worker-oriented methods: high specificity vs moderate specificity, describe in explicit terms (most basic descriptive unit of work activity, smaller than task), made up of action verb + result of action + tool used + degree of discretion given
• Job analysis interviews (structured / unstructured) - collect job information / serve as means for clarifying or verifying information previously collected / method for collecting relevant job data, \ not cost-efficient, distortion of information, lack of standardisation (both good and bad); typically with incumbents + supervisors (range of SME for better coverage)
• Critical job tasks: action verb + object of action + expected output + equipment used
• Application: formulate plan (objectives clear, interviewees known, questions and means for recording answers clearly specified, interviewers identified); steps: build rapport, make purpose clear -> steer according to logical sequence of duties, open-ended questions (show no partiality, no attempt to suggest improvements) -> summarise
• Job analysis questionnaires (tailored / prefabricated or generic) - lists job information for raters to judge how much it applies, efficient means of collecting from large numbers \ costly, motivation problems with long inventories, respondents must be representative of workforce; completed by incumbents
• Task analysis inventory: contains background information + ratings given on rating scale eg frequency / importance / difficulty + miscellaneous information
• Critical incident - based on direct observation / memory / interview, describing incidents of work behaviours (purpose of act seems clear to observer + consequences sufficiently definite), distinguish successful vs unsuccessful work performance, basis for developing descriptive information, behavioural-based \ too contextualised, labour intensive; developed by supervisors + other SME
• Useful account of critical incident: focuses on specific incident + identifies behaviour + context + consequences
• Application: select method + select panel of experts + gather critical incidents + classify into job dimensions
• SME / job expert workshops = groups of 10-20 incumbents working with group leader to produce job analysis
• Preparation: select and prepare SME -> rate importance of job task -> rate importance of work-related characteristics -> link important work-related characteristics to job tasks -> develop selection plan + job content relevance (content validation)
• Multiple job analysis methods

Identification of employee specifications

• Direct vs indirect means: larger inferential leaps (rate importance of work-related characteristics on survey for entire job) vs specific steps involved in deriving critical work-related characteristics from job tasks; more often use indirect means
• Determine relative importance of work-related characteristics: SME might complete survey, questionnaire might list essential work-related characteristics previously identified (to be assigned relative importance weight) -> multiply importance ratings * task importance ratings
• Sequential steps to determine KSA (indirect)
• Identify work behaviours: what + why + how
• Rating work behaviours
• Specify necessary KSA (select and prepare panelists for rating KSA)
• Rating importance of identified KSA
• Identify other specifications necessary
• Link KSA / other specifications x work behaviour
• Developing content areas of selection procedure

Developing selection procedure

• Developing selection plan = specify relative KSA weights + choose appropriate procedures for measuring
• Determine relative weight of employee specifications (KSA importance * task importance) - indicate extent of KSA coverage that should be present in selection measure
• Choose selection methods to assess
• Questions: past behaviours / experiences prior to taking the job associated with successful performance? Means of simulating job performance? Written test best for eliciting desired reactions and providing practical scoring? Important to express themselves orally to cover job requirements? Reliability and validity of data? Feasibility?

Competencies

• Competencies = clusters of important work-related characteristics + work behaviours required to perform to job (in as fewer terms as possible, clear enough for behaviour to be recognised in action), allow for broad range of abilities, linked to meaningful outcomes describing the way people must perform at work
• Competencies vs competence: behaviour vs skill, manner of behaviour vs standard attained, how standard measured vs what is measured
• Competency framework = documentation that defines behaviours desired, provides common language and framework for integrating HR systems / guide for performance measurement; simplifies job analysis information into key components, creates strategic linkage to business strategies
• Competency modelling: going from job analysis -> building model; require appropriate definition + descriptor for various competency levels + desirable behaviours + key tasks and related training
• Review of information provided by job analysis: core competencies distinguishing high vs poor performing employees, repeatedly identified competencies determined from job analysis, clear definitions of each competency broken into development stages
• Requirements for implementation: simple documentation linking competency model x other HR practices, small number of competencies (~6-8), differentiation between competencies ('dictionary' for competency labels with appropriate definition)
• Downsides of competencies: value of competencies hard to define, broad description too generic to measure with accuracy (need for layer of complexity), no best practice

Position descriptions

• Position descriptions = description of tasks and responsibilities, consist of organisation + title + location + key relationships + working environment description
• Requirement of statements: discrete identifiable aspect, 1-3 sentences, outcome-based (with performance indicators), contain verb + object + purpose
• Focus on primary, current, normal daily duties

Role of measurement in selection

• Definition of measurement: systematic application of rules (same conditions, scored in same manner) for assigning numbers to people to represent quantities of attributes or traits (not directly observable, must be inferred from observable score, numbers cannot be made objective)
• Criteria (definition of employee success @ job, supervisory ratings are more frequently used) / predictors (measures of attributes identified through job analysis as being important for success); both must be relevant to job + must have appropriate ways to measure
• Individual differences - basic law of psychology
• Scales of measurement - means by which individuals can be distinguished from one another on a variable of interest
• Nominal - do not indicate how categories differ
• Ordinal - only yield 'greater than' vs 'lesser than' conclusions; best adhere to this assumption
• Interval - can assign meaning to differences between numbers; most constructs use interval scale with normal distribution
• Ratio - presence of absolute zero; \ not applicable to psychological measurements

Standardisation in measurement

• Standardisation = systematic application of pre-established rules / standards for assigning scores to attributes / traits of individuals; useful for distinguishing people on meaningful ways
• Characteristics
• Content: measure by same information, including same format / medium
• Administration: same way @ all locations, across all administrators each time applied
• Scoring: rules specified before administering, applied same way with each application; rater training can ensure inter-rater agreement

Measures in HR selection

• Validation study: criteria employed as part of research study to determine which selection procedures are related to job success; predictors employed by managers in decision making (direct impact)
• Predictors / selection procedures
• Background information
• Interviews
• Tests
• Criteria / measures of job success
• Production data
• Personnel data
• Judgement data
• Work sample data
• Job proficiency data

Developing selection measures

• Finding and constructing measures
• Locating existing measures
• Advantages: less expensive, existing idea of reliability / validity / other characteristics via technical manuals, existing measures often superior to in-house measures
• Information sources: text and reference sources / Buros' mental measurements yearbooks / BPS psychological testing centre
• Other reference sources: journals / test publishers / professional associations
• Minimum recommendations: completely understand attribute + best means (understand relationship with nature of jobs), read reviews and evaluations, study available specimen, ask for compelling arguments
• Constructing new measures - best to work with a professional
• Analyse work, specify KSA assessed by predictor
• Select measurement method
• Plan + develop measure: specifiy purposes + nature of population + how to score KSA+O + sampling designs and statistical procedures
• Administer + analyse + revise preliminary measure onto sample from same population
• Determine reliability + validity
• Interpreting measure - may develop cut-off scores, select norms
• Monitor performance

Interpreting scores on selection measures

• Norms
• Norm referencing = establish relative position against group of others
• Need to know: how others scored, validity of selection procedure
• Guidelines: norm group must be relevant (same norm group for all), local norms where appropriate, transitory
• Percentiles
• Percentiles = % of persons in norm group falling below given score
• Standard scores
• Standard scores = adjustment to raw scores -- possibility to determine proportion of people @ various standard score levels

Theories of test design

• CTT vs IRT: all items same points vs harder items more points, overall items correct vs overall points
• IRT - focus on item performance, each item has ICC describing probability of getting right vs wrong given ability of test taker (seen as stable, can be measured on standardised scale)
• Particularly popular @ domain of maximum performance test \ not very frequent in personality psychology
• Basic idea of IRT: individual response determined by trait level + qualities associated with item (difficulty / discrimination etc)
• Perfect ICC with perfect discrimination: probability of correct answer rises instantly from 0% to 100% as soon as ability of respondent meets difficulty
• Item delivery methods
• Adaptive testing (subset of IRT) vs classical testing: difficulty tailored to ability level vs fixed set of questions, increased test security (lower risk of overexposure to items), fewer items needed to achieve accurate estimate (less time to complete, reduced candidate fatigue, more accurate scores)
• Item banks (can be for both adaptive / classical test) = repository of test questions, can be sorted by item content / test difficulty to generate pool of equivalent items within each (randomly selected which one to proceed); IRT measurement system can calibrate items to establish equivalence

Reliability

• Reliability = dependability / consistency / stability of scores on a measure (between >2 sets of scores)
• Errors of measurement - proneness to error due to sample / test taker / examiner / situation factors
• True score: individuals answered correctly same % of problems they would have if all possible problems given, + test was construct-valid for underlying phenomenon, + individuals answered correctly without being affected by external factors eg emotional state / physical health
• Error score: factors affecting obtain scores not related to attribute, eg individual's mood / person administering the measure / scorer judgement / physical conditions
• Reliability coefficient = index of relationship
• Questions to determine choice of method: how dependably can individual be assessed @ given moment? How dependably will data collected today represent same individual later? How accurately will scores represent true ability on trait? Evaluations vary from one rater to another?

Methods of estimating reliability

• Test-retest
• Assumptions: construct is stable, errors can be different \ variance of errors equivalent over time
• Factors affecting test-reteset reliability: memory (mitigated by increasing time interval, more items, more complexity) / learning (if time interval too long; lead to systematic \ different changes among respondents -> decrease reliability)
• Parallel forms - used to avoid reuse of measure \ different items of same attributes administered @ later time
• Same assumptions as test-retest + equivalent forms exist (\ test content may be prone to error)
• Internal consistency: shows extent to which data free from error due to the way items are phrased / interpreted by respondents etc
• Split-half - reliability underestimated; corrected via Spearman-Brown; not appropriate for time-limit tests
• Kuder-Richardson - only appropriate for dichotomous items
• Crobnach's alpha
• Inter-rater reliability: error variance due to use of different raters to provide scores on same target
• Inter-rater agreement - indices generally restricted to categorical data
• Interclass correlation: >2 raters make judgements about series of targets / objects being rated
• Intraclass correlation - reveals average reliability of 1 rater + average reliability of all raters (>3)

Interpreting reliability coefficients

• Reliability coefficient - indication of proportion of total differences attributable to true differences
• Considerations of examining particular reliability coefficient
• Specific to reliability estimation method and group
• Necessary \ not sufficient for validity
• Based on responses from a group (not person), determined ultimately by judgement
• Deciding how high - no agreement; higher reliability needed for more critical decisions (especially @ predictor level when hiring key personnel), consider specific circumstances to determine how much error is bearable
• Rule of thumb: internal consistency for cognitive ability @ .80 (lesser different ways to look at reasoning), internal consistency for personality traits @ .70, test-retest / equivalent forms / inter-rater reliability @ .70
• Practical applications of standard error of measurement - @ individual score level, based on perspective that measures are approximations represented by band / range of scores; can aid decision making where only 1 individual involved
• Aiken's guidelines: difference between 2 individual scores only significant if difference >2x standard error measurement; difference between 2 measures on same person only significant if difference >2x standard error measurement of either
• Evaluating reliability coefficients: must search for reliability + validity data (including for certain subtests / forms) + normative data -> systematically evaluate

Factors influencing reliability

• Method of estimating (parallel forms reliability long-term < parallel forms immediate < KR / alpha < test-retest long-term < split-half < test-retest immediate)
• Individual differences: greater variability or standard deviation --> higher reliability
• Stability
• Sample: representativeness depend on choice + number of respondents, similarity to population
• Length
• Test question difficulty: moderate difficulty produces higher variability
• Homogeneity of content
• Response format: increase number of possible scores for a measure up till a certain point
• Administration / scoring

Validity

• Validity = degree to which available evidence supports inferences from scores; applies to selection procedure + predictor + criteria
• Relationship with reliability: maximum possible correlation (validity coefficient) = sqrt [reliability of predictor * reliability of criterion]
• Validation study = provide evidence for determining inferences that can be made from scores

Content-related validation strategies

• Content validation - relates to collection of job + KSA necessary for effective performance; more of descriptive validity
• Reasons for content validation
• Adequate amounts of predictor / criterion data not available (especially in small businesses)
• Adequate measures of success criteria not available, eg due to cost
• Maximise correspondence between job content x selection procedure -> enhance applicants' favourable perceptions
• Difference with other validation strategies: prime emphasis on construction of new measure, emphasise role of expert judgement
• Major aspects
• Comprehensive job analysis: description of tasks + importance of task + KSA required + importance of KSA (including operational definition + relationship to each task + complexity of obtaining + specification on prerequisite) + link to task
• Select experts - should also provide how SME made judgements in validation study (eg individual vs group consensus)
• Specify measure content (domain sampling) - to be reviewed by SME to judge suitability for job, test for fidelity (comparability between content x performance domain) in phsyical / psychological / job<->KSA aspects
• Assessment of selection measure x job content relevance

Limitations of content validation

• Issues of physical and psychological fidelity
• Does successful performance on measure require same KSA for successful job performance?
• Mode used for assessing test performance same as required for task / job performance?
• Any uneccessary KSA in the predictor?
• Situations where content validation is inappropriate
• Constructs not directly observable but inferred from selection device, eg judgement / integrity / motivation / mental processes -- relevance to job performance?
• Employee expected to learn KSA on the job
• Content does not resemble work behaviour / setting of selection procedure does not resemble work setting
• Content vs criterion validation: focus on selection measure vs external variable, broader base of data vs specific data set, jugemental descriptors vs quantitative predictor indices

Criterion-related validation strategies

• Concurrent validation - simple correlation with p < .05 for significance
• Steps: job analysis -> determine relevant KSA -> experimental prediction of KSA -> select criteria of success -> administer to current employee -> analyse predictor x criterion relationship
• Factors affecting usefulness of concurrent validation: differences in tenure or length of employment (experience may affect job performance) / representativeness of present employees to job applicants (eg due to job experience, effect of turnover or dismissal) / missing individuals from validation study / motivation to participate (different between applicants vs employees) / manipulation of answers
• Predictive validation - gold standard, like a longitudinal measure
• Same steps as concurrent \ except administer predictors to applicants
• Appropriate inference of how well job applicants can perform on job \ time interval issues, also difficult to explain to manager on importance of filing before use of data for selection
• Concurrent vs predictive: similar usefulness in reality
• Requirements for criterion-related validation study: jobs not in a period of change, relevant reliable criterion without contamination, must be possible to base on representative sample for generalisation, large representative sample
• Court perspectives: may prefer to judge validity on content or format, preference for predictive vs concurrent, more inclined to listen to experts, judges can differ on willingness to accept statistical corrections)

Empirical considerations in criterion-related validation

• Important questions: relationship between responses on measure x job performance? Strong enough relationship?
• Correlation
• Computing validity coefficients: index summarising degree of relationship between predictor x criterion -> squared to obtain test's ability to account for individual differences (clients want to know this)
• Importance of large sample sizes: lower threshold of validity coefficient for statistical significance, more reliable magnitudes, greater statistical power
• Interpreting validity coefficients: square validity coefficient to obtain coefficient of determination; can also use expectancy tables / charts / utility analysis
• Prediction - show success for group \ not individuals
• Linear regression: mathematical equation that describes functional relationship between predictor and criterion
• Simple regression
• Multiple regression
• Cross-validation (empirical / formula): test for shrinkage on another sample
• Empirical cross-validation steps: group randomly divided into two -> regression develop on one of the two -> predict criterion of other group -> predicted scores @ holdout group correlated with actual criterion scores; or use independent sample
• Expectancy tables / charts: divide group into superior performers vs others -> frequencies determined for each predictor score -> predictor score divided into fifths -> expectancy chart to depict percentages of superior performers

Construct validation strategy

• Construct = postulated psychological concept / attribute / characteristic / quality thought to be assessed by measure; continuously accumulate empirical evidence, represent much broader definition of validity than single criterion / content validation
• Steps in implementing construct validation study
• Constructs clearly defined with hypothesis formed about relationship with other variables
• Develop measure to attempt to assess construct
• Develop studies testing hypothesised construct x other variable relationships
• Guidelines helpful in construct validation
• Intercorrelations - should show nature of groupings
• Parts of measure belonging to same grouping should be internally consistent
• Different measures assessing same construct should be related with developed measure (convergent), measures assessing different constructs should be unrelated (divergent)
• Content validity should show how experts judge how parts of the measure were developed + how these sampled job content domain

Factors affecting size of validity coefficients

• Reliability of criterion / predictor - can adjust validity coefficient with correction for attenuation
• Recommendations: correct validity coefficients for inter-rater reliability (assumes accurate rating information on participants collected from >1 supervisor), may need to collect rating from peers if only 1 supervisor, otherwise correct based on meta-analytic estimates of inter-rater reliability / compute alpha (tend to overestimate reliability --> more conservative estimate)
• Restriction of range (direct / direct): lack of individual differences
• Criterion contamination - controlled by partial correlation / adjustments to criterion data eg computation of ratios
• Violation of statistical assumptions, eg if non-linear

Utility analysis

• Utility analysis = summarises overall usefulness of selection measure, using dollars-and-cents + other measures eg percentage increases in output --> shows degree to which use of selection measure improves quality of individuals selected vs not using selection measure
• Factors determining percentage of employees identified successful following use of valid test
• Validity coefficient: correlation of test x criterion
• Selection ratio: ratio of hires :/: applicants
• Base rate: percentage of employees successful on the job without use of test; low base rate indicates difficulty with current selection system to identify successful employees
• Brodgren-Cronbach-Gleser model: utility change = number * average tenure * average z-score * correlation * dollar value of standard deviation (usually 40% salary) - cost
• Acceptance by practitioners?

Other statistical methods

• Binomial effect size display - displays practical importance of effect without relying solely on r or r^2; very simple description
• Calculating BESD: success rate .50 + r/2, failure rate .50 - r/2; r = predictive validity (difference between odds ratio)
• Taylor Russell tables: estimate of incremental validity of test; very old school
• Find base rate of success + find selection ratio + find validity of selection tool -> expectation on how many good performers; useful when selection ratio low +/ validity high
• Bayesian statistics: modifying conditional probability, represent unknown fixed parameter as random variable @ prior distribution -> updates to posterior distribution using Bayes' rule (information from new data + prior knowledge -> updated probability); allows evidence to be monitored as data accumulates, eg in case of multiple hurdle selection (probability of success may grow over stages) / probability change in subsequent stages after more information is added after cognitive ability test; \ might require candidate to keep coming back

Validity generalisation

• Validity generalisation = statistical analysis of information accumulated from multiple validation studies involving a predictor -> results combine to determine overall effectiveness in new settings
• Factors influencing differences among validity coefficients reported in validation studies: sampling error (small sample sizes) / difference in predictor reliability / difference in criterion reliability / difference in restriction of range / criterion contamination and deficiency / computational and typographical errors / slight differences among tests thought to measure same attributes
• Methods of validity generalisation: obtain large number of published vs unpublished validation studies -> compute average validity coefficient -> calculate variance of difference -> subtract variance due to effect of small sample size -> correct average validity coefficient and variance for errors due to methodological deficiencies -> compare to determine variation in study results --> small differences attributable to methodology hence can generalise
• Conclusions from validity generalisation studies: no need for every organisation for every job (no organisation effects), mental ability test as universal predictors
• Criticisms: corrective formula based on hypothetical values, may overestimate amount of variance attributable to key study deficiencies
• Validity generalisation requirements: show that selection procedure assesses same work-related characteristics / representative example of measure in database + show that job in new employment setting == jobs included in database

Job component validity

• Job component validity = standardised means of obtaining detailed information on jobs for which validation study is being conducted; underlying hypothesis: validity for common job component consistent across jobs
• Synthetic validity = inference of test battery validity from pre-determined validities of tests for basic work components; know key performance of job -> determine test validity for these components -> combine validities into whole
• Important assumptions: same KSA of common work component apply across the jobs, predictor's validity for KSA required for work component consistent across jobs
• Conducting job component validity study
• Position analysis questionnaire (pen-and-paper) - comprehensive listing of general behaviours @ work
• Identify major components
• Identify attributes (may use ratings from experts)
• Choose tests that measure most important attributes identified
• Accuracy of job component validity studies: similar across components + with validity coefficients in actual validation studies
• Criticisms: less useful in predicting actual validity coefficients (especially psychomotor test data)

Future of validation

• Validation options for small business: content validity / validity generalisation / job component validity / other synthetic validity options
• Changes in workplace
• Increasing numbers of small organisations without skills to undertake traditional validation strategies
• Increasing use of teams
• Changes in definitions of job success (eg commitment / teamwork / quality)
• Job requirements more fluid -- require focus on broad work capabilities
• Balance between competing issues: technical requirements of valiation research vs changing nature of organisation needs vs societal demands vs legal implications

Strategies for selection decision making

• Basic varieties of selection decisions: simple (1 position, several applicants) vs complex (several positions, several applicants)
• Information processing demands = number of applicants * amount of selection data
• Selection decision errors: false positive (erroneous acceptance; may need remedial training / probationary period to reduce long-term cost) vs false negative (erroneous rejection; may work for competitors)
• Questions on employment decision making: best methods for collecting predictor information @ specific situation? How to make decisions based on overall score?
• Methods for collecting / combining predictor information
• Mechanical vs judgemental data collecting: data collected without human judgement vs use of subjective judgement
• Mechanical vs judgemental combining: statistical equation (pure statistical / trait ratings) vs gut instincts (profile interpretation / pure judgement) to arrive at overall impression
• Mechanical vs judgemental composite: combine judgemental + mechanical data using formula (typically @ AC for promotion) vs made with overall judgement; *mechanical composite usually superior to all methods
• Mechanical vs judgemental synthesis: combine all data (either judgemental / mechanical) using formula vs made with overall judgement
• Advantages of mechanical vs judgemental approaches: proper weighting for accuracy of prediction, additional data can be added to available data to improve prediction, reduce bias / overconfidence effects, consistency
• Implications for decision makers: minimise role of judgements, standardise procedures, avoid relying on preliminary information

Strategies for combining predictor scores

• Multiple regression: equation weighs each score to arrive @ total score (possibility of identical score from different individual predictor scores), regression weights determined by each predictor's influence in determining criterion performance; gold standard
• Assumptions: predictors linearly related to criterion, with additive / compensatory effect
• Advantages: minimise error in prediction, flexible, can be modified to handle nonlinear relationships, can use nominal data; \ disadvantages: lack of stability of weights in small samples (require cross validation), may mask people who don't meet baseline, costly on large applicant pool
• Unit weighting: raw scores converted to z scores -> assign different weights; predictors identified to maximise relationship with criterion; perform better with sample sizes <75
• Multiple cutoffs: reject if any one of predictor scores fall below minimum cutoff
• Assumptions: non-linear relationship, non-compensatory
• Advantages: narrows applicant pool to smaller subset minimally qualified, conceptually simple; \ costly, only identifies applicants minimally qualified
• Multiple hurdles: go through predictors sequentially, must not fail cutoff @ any stage
• Double stage strategy: variation of multiple hurdles using two cutoff scores, with provisional acceptance between C1 and C2 subject to additional setting
• Advantages: same as multiple cutoffs, + less costly; \ reduces application pool to high end -> restriction of range, also increased time to implement
• Combination: multiple cutoff -> multiple regression (assume compensation can occur above minimum cutoff, can rank-order)

Approaches for making employment decisions

• Top-down selection: rank-order scores from highest to lowest
• Assumptions: higher score --> perform better (maximum utility)
• Problems: adverse impact against racial / ethnic groups (especially using mental ability)
• Cut-off scores: reject applicants scoring below cutoff
• Ways of setting cut-off scores: judgement of SME (Ebel / Angoff / contrasting groups) / based on how applicants (or others eg incumbents) perform on selection procedure (empirical testing procedure)
• Problems: adverse impact (not as bad as using top-down)
• Banding: establish ranges where scores within the range treated the same; can address adverse impact
• Rationale: scores have measurement error (not perfectly reliable)
• Other means / attributes used to choose within a band, eg interpersonal skills / experience
• Statistical methods: standard error of measurement / standard error of differences (standard error of measurement * sqrt[2])
• Fixed vs sliding bands: top score minus constant band widths vs band recalculated with next highest applicant score after selection
• Advantages: flexibility, ability to identify minority groups, use of secondary hiring factors; \ disadvantages: economic utility, arbitrary justification of meaningful cutoff, dependent on distribution of score, legal issues

Practical approach to making selection decisions

• Considerations when choosing among procedures: sequential vs non-sequential? Compensatory vs non-compensatory? Ranking vs banding?
• Assess job / nature of job performance: number of applicants? Cost? Time?
• Criteria necessary for building mechanical selection model: same decision made repeatedly, data available on outcomes of past decisions, good reason to expect future resemble past
• Subjective linear model: multiply weights attached to selection activity * rating to applicant -> summing products -> overall score to derive judgement of probability
• Bootrstrapping: regression analysis -> infer weights used by decision maker; based on assumption that people are not able to articulate how they made sound judgements
• Fairness: face validity / solid evidence for justification

Recommendations for selection decision making

• Auditing
• Use reliable + valid procedures for collecting information on job applicants
• Managers can participate in data collection process \ not combine scores or make decisions based on intuition
• Train managers to make systematic decisions (preferably using mechanical)
• Specify in advance
• Weight of standards
• Procedures used for judging whether applicants meet standards
• Procedures for combining standard weights x ratings given -> arrive @ overall applicant score
• Modified Angoff procedure for cut-off scores
• Top-down selection if selection procedure predicts job performance
• Banding is ok \ cannot use minority status alone for selecting within bands
• Decide on standard for defining successful vs unsuccessful hire -> keep track of hit vs miss (\ cannot track effects of false negatives)
• Periodically audit selection decisions to identify areas of improvement

Combining tools

• Psychometric assessments + other supplementary predictions routinely used by AC --> trait-level + task-level information helps to build multifaceted profile of individual
• Overlap problem: too high correlation for predictors -> add depth \ does not add breath or extra value
• Incremental validity: ability to add prediction of performance beyond other predictors
• Compiling data (typically wash-up @ AC): discussing scores of each candidate with other assessors to reach final decision, one rating form per exercise; important to be aware of biases, use evidence of behaviour when discussing
• Importance of compiling data: make accurate employment decisions, identify strengths + areas of development (use matrix with qualitative feedback)
• Problem with integrating data: may have task effects --> differences not so meaningful to combine

Other issues involving selection decision making

• Tradeoff between predictive power vs explanatory power --> importance of other validities, eg face (important for buy-in and perceived fairness) + content + construct (relationship with desired outcomes)
• Adverse impact: difference in success rates for different groups, occurs in subgroup differences, becomes bias if unrelated to job performance (unjustified)
• Affirmative action: provide specific mechanisms for reducing under-representation + improving opportunities of particular demographic groups discriminated, eg quota system (qualified people ranked from most qualified to least qualified, likewise for qualified minorities -> top few decided based on quota) / preferential treatment

Application forms and biodata assessments

• Application form = series of questions designed to provide information about general suitability of applicants; starting point
• Role of application forms: decide whether applicant meets minimum requirements of position + assess and compare relative strengths and weaknesses of individual
• Biodata = application information developed / scored in a way to maximise prediction (scoring keys note specific individual application distinguishing between successful vs unsuccessful employees); excludes directly assessing personality traits etc
• Weighted application blank = sum of collective weights for responses to items
• Biographical information banks = specific type of biodata based on notion that deeper study of applicants + backgrounds can be effective predictors
• Ways to categorise biodata: response-type // behaviour-type
• Dimensions of biodata items: verifiability / historical vs futuristic / actual vs hypothetical / memory vs conjecture / factual vs interpretive / specificity / response vs response tendency / externality of event
• Legal implications: cannot include questions that disproportionately screen out minority groups / cannot be justified to predict successful performance
• Appropriateness of questions: adverse impact? Information necessary to judge competence (content validity)? Invasion of privacy? Responses to question associated with success vs failure? Fair employment laws?

Evaluating application blanks as predictors

• Validity generalisability guidelines
• Sound reasons to expect validity of biodata instrument to generalise to other people / situations?
• Validity of criterion?
• Validity of item? Reduces sample dependence of instrument if empirical and conceptual justifications exist
• Adequate representative sample?
• Prediction of job performance: biodata can predict broader aspects of performance better than mental ability \ can predict only when criterion is taken into account when identifying which items to rely on
• Issue of distortion - must warn applicants that false information may be grounds for dismissal, explicitly inform that information will be checked, have applicants sign a statement certifying accuracy of information

Developing application forms

• Composition of application forms
• Instructions for applicants - clear, understandable, fair, easy to use prohibit giving unsolicited information, address privacy concerns, state that form has been reviewed, state that disabled applicants can request reasonable accommodation
• Questions for applicants - job-related questions only, no personal information (other than that necessary for initially judging qualifications), no discriminatory questions (also makes applicants less likely to accept offer); to make sure person is suitable
• Selecting application form content - application should be shorter less-detailed for lower levels, possible questions identified via job analysis \ must be reviewed for fairness / usefulness
• Developing and revising application forms (consistently needs to be updated)
• >1 application forms (because many types of jobs)
• Job analysis data as basis for choosing questions; behavioural items more likely to provide reliable data than ratings
• Pool of biodata items from publications / from empirical analysis
• Guidelines of formatting biodata items: deal with past behaviour (to be weighted later), unique discrete events, try to limit to firsthand recollections, specific, brief, escape option given, no family or personal matters
• Pre-screening / pilot-testing: review items by panel of judges -> remove items (eg little response variance / skewed distributions / correlate with protected groups / no convergent validity / no criterion validity etc) -> test on large group representative of applicant population
• Scoring - may need unit weighting
• Single score via empirical keying, eg vertical percentage (weighted application bank) / horizontal percentage / correlation / deviant response / rare response / differential regression
• Multiple scores for dimensions

Resumes

• Resume - first impression for professional positions; \ subject to same kind of distortions as applications
• Resume fraud indicators: inflated educational credentials / inconsistent periods of employment / gaps in time period listed / claims of self-employment / regressive work history / use of qualifiers / vague answers
• Attributes inferred from resumes: educational achievement -> language and mathematical abilities, sport participation -> physical abilities, authority position -> leadership attributes, etc
• Online screening - automated review can be via scanning and scoring using keywords
• CV parsing technology: electronically gathers + organises + automates scanning via keyword search, eg contact information / educational background / relevant skills
• ATS: streamlines recruitment process via maintaining database from recruitment process to keep track of openings and applicants, including matching candidates to jobs
• Legal issues with online screening: lack of internet access, privacy concerns

T&E evaluations

• T&E = rationally assess previous experience / training / education information on any questionnaire (usually @ biodata) --> evaluated by rater via scoring plan, either as sole basis of decision / means of rank-ordering individuals / pre-screening prior to administering more expensive predictors / in combination with other predictors
• Rating of T&E: consider which aspects judged relevant by raters -- measured by frequency / length of time (usually levels off after 5 years) / challenge (continue to increase competency / self-efficacy) / complexity / quality of instruction or feedback / relevancy (in terms of fit / criticality)
• Reliability: high inter-rater reliability
• Validity: less looked into; generally low correlationvalidity varied with type of procedure used (behavioural consistency method highest correlation with GPA @ .45)
• Useful information from T&E: experience -> more opportunities to learn in the past / acquire job knowledge faster -> greater job performance \ levels off at some point
• Legal implications: avoid collecting information that disproportionately screen out members of minority group, especially if does not predict successful performance / not related to requirements of job / not justified as business necessity

Methods of collecting T&E evaluation information

• Characteristics of T&E evaluations: listing / discription of KSA + other relevant job content areas, applicants can rate extent of training / experience, basis for scoring self-reported T&E
• Holistic judgement: cursory review, general judgement of suitability; informal, unstructured, can vary from one evaluator to another (unknown reliability / validity)
• Point method: points assigned based on recentness / amount of experience / amount of education; pre-established rating system determined through job analysis; lacks validity
• Grouping method: divide applicants into groups that represent levels of qualification (depend on particular situation); combinations chosen to define groups should distinguish among groups in terms of job success (determined by SME), no ordinal ranking
• Behavioural consistency: applicant describes achievements related to job competencies; formally scored with scales developed from SME, assumes past performance best predicts future success
• Task-based: applicant rates critical tasks listed on form whether performed the task before; may need verification
• KSA-based - same as task-based

Recommendations for using T&E evaluations

• Minimum qualifications rather than selection standard
• Replace holistic method with competency-based approaches
• Uniform guidelines (conduct validation studies on job relatedness)
• Only use as rough screening procedures if previous experience and training are necessary for job performance
• Standardise forms / procedures for collecting / scoring
• Data verification of self-reported data
• Final hiring decisions based on other selection measures to minimise risk of distortion of self-reported data

Reference checks

• Purpose of reference checks
• Verify information given by applicants -- basis for negative selection
• Predict job success / screen out unqualified applicants (assume past behaviour predicts future performance)
• Uncover background information
• May be used as last resort to decide between two applicants
• Types of reference data
• Employment / educational background
• Appraisal of character / personality
• Estimates of job performance capabilities
• Willingness of reference to rehire
• Reliability and validity: little research on reliability; low validity (affected by multiple factors, eg applicants preselection of who will evaluate, not enough opportunity for referees to observe) \ potential to increase predictive validity if greater structure / systematic assessment ratings from others familiar with applicant's performance with previous work situations
• Applicant acceptability: vary depending on intrusiveness of information; references > cognitive ability evaluations, references < interviews / work samples

Legal issues of reference checks

• Discriminatory impact
• Defamation of character (unique to reference checks)
• Conditions for defamation: written / oral defamatory statement + false statement of fact + injury occurred to applicant + previous employer have no absolute / qualified privilege
• Negligent hiring: third party (eg client) sues employer for injuries caused by employee
• Conditions for employer liability in negligent hiring: injury to third party shown caused by employee + employee shown unfit for job + employer knew employee unfit for job + foreseeable outcome of injury received by third party + reasonable outcome of what employer did / did not do in hiring the employee
• Types of issues: intentional employee misconduct / physical harm / unqualified to perform skill; usually involving jobs that involve public safety / individuals receiving personal care / access to others residence
• Encouraging former employers to provide information: submit written request for information on specific questions + include release form signed by applicant (read and approve information requested + request information given); if employer refuses --> call HR person to ask why / ask how request should be made to be honoured; if still refuse --> inform that failure to cooperate is being documented, at minimum verify statements about position held + years etc, (if needed) state that applicant will be told previous employer's refusal was the reason

Methods of collecting reference data

• Telephone checks (most likely to be used) - fast, allows follow-up questions, elicits information which may not be put in writing
• Fax / internet / email - may be necessary when searching public records
• Mail checks (written questionnaire); \ low return rate
• Letters of recommendation - restricted to high-skill jobs \ may provide greater depth, rare negative information
• In-person checks - high-level of interaction -> more useful information (often part of background investigations for jobs with potential security / financial risk)
• Conditions of useful data: referee had a chance to observe candidate in relevant situations + have knowledge of candidate + competent to make evaluations requested + frank and honest assessments
• Sources of reference data
• Former employers - particularly valuable for verifying past employment + evaluating work performance
• Personal references (*unlikely to be asked) - important to ask how long + in what capacity referee knows applicant
• Investigative agencies - conduct background checks, focus on application informations \ expensive
• Public records - justified by nature of job (often for those with high degree of public contact / involving personal care / with access to others belongings), caution to ensure non-discrimination
• Federal court records (suitable for higher level jobs) - civil court case information, where applicant lived / worked etc
• Educational records - may need authorisation signed to authorise release of educational records
• Competency-based questions: structured reference check directly linked to competencies required (premise of past behaviour x future performance), follow STARR for specific examples of observed behaviour

Recommended steps for reference checks

• Involve job-related concerns (emphasis on KSA that distinguish effective vs ineffective performance)
• >1 general form to cover all positions (because content only tailored to specific jobs)
• Uniform guidelines (fairness + validity)
• Structured checks on factual / behavioural data
• Applicants given written permission to contact references
• Training on how to interview referees -- for systematic / structured approach
• Record in writing
• Ask applicant for additional references if reference information cannot be obtained
• Check all application forms / resume information (pay attention to educational background / pre-employment records)
• Verify accuracy of negative information + ensure that disqualification is on the basis of success vs failure + use same information consistently for all applicants

Selection interview

• Interview - agenda-driven social exchange; also provides opportunity to see how socially competent an interviewer is
• Challenges of social exchange: competing interests in the interview -> portray image to fulfil self-interests
• Advantages of interview: opportunity to recruit good candidates + sell job, applicant to assess job fit, efficient and practical for measuring many KSA, help employer screen out / screen in applicants
• Recruiting applicants
• Providing job information - use mix of selection (employer assess applicant's qualifications) + recruiting focus (increase applicant's willingness to accept offer); alternative to communicate via written job description
• Recruiting outcomes: recruiter as part of information used by applicant to assess organisation (significant impact for skilled and trained recruiters \ no impact for demographic characteristics); \ effect of recruiter relatively modest
• Measure KSA: focus on a few KSA; \ also include other objectives: train interviewers about rapport building, begin structured interview after telling applicant what to expect -> keep good notes -> put away after all questions asked / answered
• Make selection decision
• Validity: situational interviews xx job knowledge / cognitive ability, BDI xx work experience / personality
• Reliability: structured > unstructured; reliability can be improved by increasing number of interviews (increase amount of information + correct for idiosyncractic biases of individual interviewers)
• Structured (situational / behavioural) vs unstructured: disciplined method for collecting job relevant information vs subjective global evaluations (more into personality level), moderate vs low predictive validity (validity contribution of cognitive ability for unstructured interviews)
• Screening vs selection interview: assess general characteristics (eg credentials) vs work-related characteristics

Interview types

• Structured vs unstructured interviews: same order vs free-flowing questions, common rating scale (gives most structure) vs no standardised rating scale, interviewers agree on acceptable answers vs no need for agreement
• Unstructured interview - rich set of interview skills applied consistently \ not necessarily identically, listen and reflect more
• Advantages: applicant has more discretion in answering questions, chance to learn more about job, interviewers prefer flexibility to get to know the person (natural bias), can be responsive to individual differences and situational changes (key merit), more in-depth assessment of KSA with increased accuracy (especially in areas where proficiency is less evident), opportunity for selling job to applicant, better at predicting performance @ job training programmes; \ disadvantages: more open to interviewer effects, scalability problem
• Psychometric properties: more related to general mental ability, conscientiousness, emotional stability, motivation-based
• Follow-up / probing questions: descriptive (applicant provides descriptions about activities) / structural (find out how applicants organise their knowledge) / contrast (applicant discuss meanings of situations + make comparisons)
• Structured interview - behaviour-based around KSA; need to develop / deliver properly to avoid serious errors
• Factors influencing ratings: physical attractiveness, impression management behaviour, verbal / non-verbal behaviour of applicant
• Biases most relevant: anchoring / confirmation bias / illusory correlations / overconfidence (belief in ability to predict human behaviour); best avoided via training
• Rules for structuring interview: questions should be job related, ask all same questions, rate using anchored rating scales, multiple interviewers independently evaluating, train interviewers
• Screening interview (conventional): general KSA identified from job analysis -> most important used -> identify questions using modified critical incidents technique (supply examples only in reference to KSA identified)

Developing structured interview questions

• Situational interview: candidates asked what they would do when faced with particular situation (future focused); based on goal-setting theory and assumption that intentions predict future behaviour
• Core aspect of dilemma - require applicant to choose between >2 exclusive courses of action
• Advantages: perceived practical and relevant by both interviewers / applicants, good for those without much experience, good at predicting future performance, data gathered integrated with other sources, measure job knowledge / cognitive ability (\ not as much as unstructured); \ disadvantages: slightly lower ability to predict job performance (especially for more complex roles)
• BDI: candidates asked to provide evidence of demonstrating certain behaviours in the past (past focused); based on assumption that past behaviour best predicts future behaviour
• Advantages: well-designed items (difficult for candidate to over-sell), accurate consistent predictions, structured systematic approach with minimal overlap / repetition, data gathered integrated with other sources; \ disadvantages: limited to small range of selection criteria, not suitable for those with limited work experience eg fresh grads (\ job experience shown not related to performance on these questions)
• Can give prompts on interviewee's role \ no leading prompts, don't give more information than another candidate
• STARR format: situation (what) -> thinking (how) -> actions (how) -> role played (level of responsibility) -> results (effects)

Discrimination in interview

• Conditions for discrimination: decisions lead to disparate treatment / impact + interview could not be defended for job-relatedness
• Research on discrimination: structured interviewers reduce demographic influence on outcomes, disclosure of nonobious disability increases likelihood of hiring, small inconsistent effect on result if interviewer vs applicant of same race / gender

Model of interview decision making

• Dipboye's model: prior information about candidate / job / interviewer / organisation / labour market -> interviewer / candidate differences in ability / motivations / expectations <-> social interaction @ interview (<-> processing / gathering of information) <-> interviewer assessment of candidate (-> validity / reliability / fairness) // candidate assessment of organisation / job / interviewer (-> candidate attraction); assume interviewer and applicant are gathering information about each other / organisation / job
• Expectations / beliefs / needs prior to interview
• Self-fulfilling prophecy: seek out / distort information that supports pre-interview impressions (problem of access to applicant information before interview)
• Image projection of candidate - less vigilant for candidates with higher expectations -> received lower ratings
• Individual differences: most affected by physical attractiveness / handshake / personality / participation in coaching program prior; \ no effect by demographic characteristics
• Social interaction during interview
• Candidate impression management can create positive / negative images, ingratiation positively affect interviewer's evaluations, self-promotion tactics positively affect interviewer's evaluations (ranked differently by different people)
• Interviewers disagree about which topics to cover in an interview (related to individual preferences)
• Information processing / decision-making factors after interview
• Two stage process: reduce choice by screening out unsuitable candidates (categorise) -> evaluate choices more rigorously (characterise)
• Information weighting: initial information carries considerable weight (anchor -> filter all subsequent evaluations), negative information (even on only 1 characteristic) > positive information, interviewers may fail to recall half of information

Recommendations for using interview

• Restrict scope - focus on just questions, core KSA, core purpose (recruitment // selection separated systematically, no need to ask biodata)
• Limit use of pre-interview data: only to complete data of core KSA, incomplete statements on application blank / elsewhere
• Structured format: set of questions formulated, may go beyond questions to clarify a response / seek other important details
• Job-related questions
• Question of job knowledge
• Question of social skills
• Question of personality / habitual behaviours - most difficult to assess accurately
• Multiple questions for each KSA - depend on information from job analysis / time available; consider relative importance
• Multiple independent interviews (*not panel interviewers) - pooling reduces influence of measurement error; same interviewers interview everyone
• Formal scoring format - commonly interval
• Train interviewer (regardless of structured / unstructured)
• Receiving information: hearing / observing / remembering
• Evaluating information (systematic scoring)
• Interviewing behaviour
• Result of training - how much improvement?

Ability tests

• Ability tests (aptitude / achievement) - measure knowledge up to time of test (except physical ability tests); all have definitive right vs wrong
• Cognitive ability tests - related to job performance, close association with academic achievement
• Types: verbal (eg comprehension / semantic relations) / mathematical (eg numerical fluency) / memory (eg memory span) / reasoning (eg visualisation / ordering / spatial orientation)
• Variety of scores obtained (score for each ability +/ overall score), one type measures separate abilities (no combining scores)
• Tests of maximum performance: attainment (current) / ability (currently capable of doing) / aptitude (what someone will learn / do in future); all are snapshots
• Theoretical perspectives of intelligence
• Spearman's g factor >- specific factors
• Cattell and Horn: divide notion of general intelligence into fluid (problem solving outside prior experience, adaptability, reasoning in abstract and pure form; not innate) // crystallised (use knowledge / skills accumulated through experience + prior learning); CHC broad ability factors (hierarchically organised): fluid / crystallised / domain knowledge / visual-spatial / auditory / retrieval / processing speed / reaction time
• General reasoning vs critical reasoning tests: overall capacity / comfort with core concept vs aptitude to work with verbal + numerical material, no context to words vs use context to make inferences, useful for understanding / proficiency vs evaluation / judgement / application
• Existing tests: Wonderlic personnel (extreme high test-retest reliability, primarily verbal comprehension -> deduction -> numerical) / Watson Glaser (moderate criterion validity with creativity, primarily critical thinking) / RANRA (analytical numerical skills) / Raven's progressive matrices (abstract reasoning) / Hogan business reasoning (tactical / strategic) / SHL verify (adaptive testing, general ability + deductive + inductive + numerical etc)

Validity of cognitive ability tests

• Validation of mental ability tests (Project A)
• Objectives of Project A: identify major domains of individual differences as useful predictor variables -> select variables appearing to be best bets for yielding incremental validity -> develop psychometrically valid measures
• Data collected over multiple years including concurrent validation + longitudinal validation sample, represent as many Army units as possible -> 65 predictor tests as measures -> 6 categories of predictor instruments, apply to 5 categories of work performance across entry-level jobs
• Uniform conclusion: mental ability tests among high validity of all selection instruments
• Validity generalisation studies
• Importance in selection: same predictor can be used across all organisation (no need construct validity studies within each organisation \ magnitude of correlation varies across jobs), scores from cognitive ability tests as good a predictor as scores from composite tests of specific ability --> no need composite tests
• Validity generalisation for same job
• Validity across jobs
• Mental ability vs other selection tests
• Cognitive ability x job performance curve: job performance start to level off after base level of cognitive ability

Discrimination in mental ability tests

• Adverse impact - large differences exist in mean scores resulting in higher number of whites being hired (can be justified if accurately predicting job performance \ not ideal); borderline acceptable - 4 :/: 5 whites :/: minorities
• Overall opinions on mental ability tests: valid and fair \ incomplete measures of intelligence, not comprehensive indicators of job performance
• Intelligence compensatory theory: conscientiousness negatively correlated with intelligence -- people with lower intelligence compensate with more conscientious behaviours?
• Effects of coaching: minimal effect on test scores \ coaching + test preparation can lessen gap between white vs minority on job knowledge test for promotion

Internet tools in mental ability tests

• Electronic delivery systems of gathering data: applications / resumes / mental ability tests / item banks
• Equivalence of pen-and-paper vs electronic - tests are equivalent \ timed tests more problematic to translate into electronic version
• Unproctored internet testing leads to: someone else taking test / use of Internet material / aid from accomplices / differences in speed vs software
• Tests most appropriate for internet testing: brief biodata / situational judgement / personality

Other types of ability tests

• Mechanical ability (manual / written) = characteristics that tend to make for success in work with machines / equipment
• Factors of mechanical ability tests: spatial visualisation / perceptual speed and accuracy / mechanical information
• Bennett mechanical comprehension test (extreme high reliability) - measure ability to perceive and understand relationship of physical forces x mechanical elements in practical situation; most widely used since historical times
• Clerical ability = for jobs requiring extensive checking / copying of words / numbers + movement / placement of objects; predominantly measured perceptual speed + accuracy in processing verbal / numerical data
• Minnesota clerical test (number checking / name checking) - brief, easily administered and scored (number right - number wrong)
• Physical ability test
• Reasons for physical ability test: increase in woman applicants for physical labour jobs, reduce work-related injuries, get around banning of pre-employment medical exams
• Physical ability analysis: static strength (maximum force) / dynamic strength (muscular endurance) / explosive strength (burst of muscular effort) / trunk strength (dynamic strength specific to trunk) / extent flexibility (trunk / back muscles) / dynamic flexibility (rapid repeated flexing trunk) / gross body coordination (coordinate several parts in motion) / gross body equilibrium (ability to maintain balance with non-visual cues) / stamina (sustain maximum effort)
• Components of physical performance: muscular strength (tension / power / endurance) / cardiovascular endurance / movement quality (flexibility / balance / neuromuscular integration)
• Legal issues: may need different standards for females / disabled / older workers to minimise adverse impact?

Recommendations for use of ability tests in selection

• Reliability data: calculated correctly on workers?
• Validity data: measure what it says to measure (via correlation with other psychological measures / factor analysis)? Test scores x measure of training / job performance (criterion)?

Personality in selection

• Personality = unique set of characteristics that define an individual + determine enduring and distinctive pattern of interaction with environment (feeling + thinking + behaving)
• Personality theories
• Freud: psyche consist of id + ego + superego; pleasure-seeking
• Jung: personality as projection, idea of persona / face / shadow; type-based theory
• Cattell: first to go into factorial analysis to get to traits (16 fundamental factors); lexical approach
• Eysenck: proponent of individual differences, especially limbic system arousal / cortical arousal -> neuroticism / extraversion
• Costa et al (Goldberg creates marker set): big 5
• McAdams & Pals: integrative science for unique variation on general evolutionary design + dispositional traits + characteristic adaptations (situational perspective) + self-defining life narratives + sociocultural context
• Cybernetic big 5: gene + environment -> stable parameters of cybernetic mechanisms -> personality traits -> life outcomes / characteristic adaptations -> environment // parameters of cybernetic mechanisms
• Other personality traits
• Emotional intelligence: self-awareness / self-regulation / social awareness / relationship management
• Core self evaluation: self-esteem / generalised efficacy / locus of control / emotional stability
• Proactive personality - action-oriented trait
• Personality x work: enhance social engagement / work motivation; conscientiousness and emotional stability valid for all jobs
• Use of personality data in selection needs specification of job tasks + identification of traits linked to tasks

Five factor model of personality

• Brain structure vs big 5: neuroticism @ mid-cingulate gyrus, extraversion @ OFC, openness @ PFC (?), conscientiousness @ lateral PFC, agreeableness @ posterior left superior temporal sulcus etc
• Higher order dimensions: alpha (stability >- emotional stability / agreeableness / conscientiousness) // beta (plasticity >- extraversion / openness)
• Stability: capacity of cybernetic system to resist disruption -> regulatory / inhibiting effects on mood / behaviour / cognition; serotonic pathway
• High vs low stability: handle disruptions or distresses well via emotional control vs frequently interrupted by emotions / impulses / doubts (even minor anomalies might plunge them into caos)
• Plasticity: tendency towards general behavioural / cognitive exploration; dopaminergic (reward-driven)
• High vs low plasticity: minor anomalies provide motivation to explore vs struggle to adapt when forced to rely on own capacity, curiosity and flexibility
• Biopsychological theory: approach (dopaminergic) vs avoidance (serotonic): pursue goals and rewards vs avoid negative events, positive emotions vs negative emotions
• Orthogonal aspects of big 5: emotional stability - calmness x confidence, agreeableness - compassionate x diplomatic, conscientiousness - drive x discipline, extraversion - sociability x assertiveness (strongly predicts leadership), openness - curiousness x abstract
• Maladaptive measures with extreme high scores: antagonism vs agreeableness, detachment vs extraversion, emotionality vs emotional stability, eccentricity vs openness, disinhibition vs conscientiousness, compulsivity vs conscientiousness

Measuing personality

• Inventories = written responses of individual as information
• Self-report questionnaires - less valid than other sources
• Five-factor model
• Forced-choice - ipsative (discriminates within individuals and variables, good for development purposes \ *not for selection, low reliability than normative measures, subject to response bias), difficult to identify faking
• Projective techniques (*cannot be used for selection) - require verbal responses to vague questions
• Miner Sentence Completion Scale
• Interview - most frequently use conscientiousness \ better for broad competencies than personality
• Observer ratings - enhance prediction of job performance, better ratings from those more familiar; cannot just infer from behaviour
• Trait vs type (pattern consisting >1 trait): broad level of categorisation with finer detail vs high level of description
• IRT - focus on item-level data, item parameters + latent traits allow researchers to improve test construction and evaluate quality of individual items
• Interpreting personality profile
• Scores show preferences not abilities; to be reported in probable not definite terms
• Inherent limitations: personality questionnaires rely on self-report (can be distorted), error prone, behaviour varies
• Stick to behaviour descriptions (not just trait labels); use manual descriptions to assist analysis
• Response style indicators: social desirability / central tendency / infrequency / acquiescence
• Take norms into consideration
• Feedback needs to be tied to job

Validity of personality traits

• Predictive validities low -- could be due to lack of common model, many tests could not get clustered under common labels
• Incremental validity high -- personality not highly correlated to other selection tools \ openness has large correlation with GMA --> no incremental validity
• Meta-analysis on predicting job performance: conscientiousness / emotional stability as universal predictors across jobs (also predicted teamwork well), extraversion / openness predicted training performance (may not be very strong), agreeableness related to teamwork

Other issues in personality tests

• No adverse impact
• MMPI cannot be used (includes scales with psychiatric disorders)
• Faking - does not affect predictive validity much; self-reports may be inaccurate for other reasons than faking
• Technology - professional standards require proctoring
• Gamification: apply behavioural theories to actions during game --> generate personality / behavioural profile; \ does not actually measure personality in the way we think

Personality tools

• DISC - mostly ipsative, has psychoanalytic components
• MBTI - use forced choice to sort into dichotomy (poor validity and reliability; no evidence that people fall into dichotomies), based on psychological types described by Jung's theory; *not suitable for selection / predicting job performance
• SHL occupational personality questionnaire (normative / ipsative): describe relationships with people / thinking style / emotions
• NEO-PIR (extreme high internal consistency for domain, moderate-to-high for facet): 5 domain + 6 facet per domain, based on factor analytic research
• 16PF / 15FQ+: intelligence / intellectance (metacognitive) in addition to 15 personality factors
• Hogan personality inventory: primary scales (eg adjustment / sociability / prudence / learning approach) + occupational scales

Recommendations for use of personality data

• Define personality traits in terms of job behaviours
• Define work effectiveness as retaining productive employees - try to focus on five factor to comprehensively measure personality (facets can be used to identify more specific traits)
• Interaction of traits x situations - assumption that traits are stable
• Powerful vs weak situations
• Relevance of work context
• Contextualise personality items to measure work behaviour; \ very contextualised, very little variance in responding
• Appropriateness of selection instrument: observer ratings > self-report, better to use instrument where appropriate responses not apparent (minimise learning of correct responses)
• Incremental validity with other selection measures, especially conscientiousness + emotional stability + GMA
• Minimise adverse impact
• Use suitable norm groups for interpretation (applicant vs non-applicant; minimise motivational distortion)
• Other uses of personality: job component validity (personality factors x attributes required to perform major components) / secondary probes (basis to corroborate evidence during later parts)

Simulation and performance tests

• Simulation tests: assess by testing situations that resemble actual parts of job being considered
• Behavioural consistency types: signs (secondary indicators of behaviour, eg work experience, unstructured interview questions) / samples (gather information about behaviours consistent with predicted behaviours, conducted via instruments / simulation exercises)
• Other types: motor / verbal / trainability
• Performance / work sample tests: earliest forms of simulations, high fidelity
• Difference with other selection devices: direct vs indirect; reduce distortion, enhance relationship
• Limitations: simulations are expensive, assume applicants have necessary KSA, difficult to construct tests representative of job activities
• Validity: consistently positive, smaller differences between whites vs minorities (\ correlate with higher cognitive abilities -- overestimation?), no complaints of inappropriateness of test (candidates can see relationships to job)
• SJT: verbal stimulations; low-fidelity (mostly descriptions of work situations than actual replications)

Situational judgement tests

• SJT (knowledge based / behaviour based) = series of written descriptions of work situations with multiple responses to each situation; cheaper than AC
• Steps of developing SJT
• Collect stories from incumbents / supervisors about situations encountered on job important for successful performance
• Review situations described, eg grouping situations, selecting representative situations, vary in difficulty / complexity
• Edit situations into item stems
• Drop inappropriate / undesirable / unfeasible responses
• Job analysis, including gathering ratings of importance / time spent on specific tasks
• Format survey
• Administer to sample of SME -> test developer reviews changes -> circulate to same SME / equivalent group regarding appropriateness of responses (look for high agreement + different appropriateness scores for options)
• Empirical validity study to account for legal questions
• Validity
• Faking

Development of performance tests

• Job analysis
• Identify important job tasks
• Develop testing procedures
• Select tasks
• Specify testing procedures
• Establish independent test sections
• Eliminate contaminating factors
• Select number of test problems
• Develop scoring procedures
• Standards
• Rules of scoring
• Train judges

Assessment centers

• Assessment center = standardised evaluation of behaviour on multiple inputs with multiple trained observers + multiple exercises; judgement made from specially developed assessment simulation, pooled by assessors @ evaluation meeting
• Advantages: rich assessment, job-related (high predictive validity), fair (high face validity), can assess group dynamics; \ disadvantages: time, people, resources, high intensity, complex design
• Exercises = devices in AC designed so that participants have several opportunities to demonstrate each KSA via behaviour
• Dimensions
• Traditional assessment devices
• Performance tests / simulation
• In-basket - designed to replicate administrative tasks
• Leaderless group discussion
• Case analysis - give specific recommendations to dilemmas
• Training assessors
• Understand behavioural dimensions
• Observing behaviour of participants
• Categorise participant behaviour (record behaviour under proper dimension)
• Judging quality of behaviour - use of rating scales
• Determine dimension + overall evaluation rating

Developing AC exercises

• Considerations: keep exercises job-relevant, guide by job analysis information, off-the-shelf vs fully tailored
• Selecting exercises: key tasks -> how to measure -> level of role -> consider context + setting
• Components of exercise design: instructions for administrator + instructions for participant + instructions for role-player + observation form + rating form (behaviour checklist for standard of performance) + participant self-evaluation form (especially @ group exercise)
• Role of assessors: observe and record behaviours + classify behaviour into criteria groups + evaluate candidates, ensure data is collected accurately, relevant data, guide candidates in what they are to do (just during introductions; minimise bias / sensitive information), make decisions to performance / outcomes
• Planning and logistics: different assessor for each exercise to avoid bias (ideally 2 assessors at a go), seating plan, ensure room space adequate

Measurement at AC

• Assessment competency matrix: link between exercises x behaviours / competencies
• Dimensions approach (take on primary role of AC): job analysis -> identify clusters of job activities important (dimensions) -> rate participants on dimensions -> make employment decisions; dimension constructs only meaningful when aggregated across different exercises
• Task approach (exercise-specific) - dimensionless, emphasise importance of exercises as substantive measures of job-relevant behaviour, stronger connection to job analysis (no over-inference about competencies)
• Task vs dimension approach: focus on job-relevance of exercise content vs identification of relevant dimensions, score on behavioural checklist aggregated within exercise + overall rating vs aggregate across exercises + overall rating, exercise effects as evidence for performance vs as method bias, observable behaviour vs underlying characteristics
• TBAC advantages over DBAC: more user friendly, less time-consuming, no adverse impact; \ lack of significant research gaps

Effectiveness of AC

• Validity: initially predictive of movement into middle-level management + reduce racial / gender differences -> same dimension shown to have poor correlation across exercises -> found overall assessment rating scores correlated consistently with various measures of work performance
• Managerial behaviour: greater proficiency in interviewing, more proficient in several aspects of communicating information, decrease in halo rating error
• Low adverse impact
• Cost
• Construct validity issues: lack of discriminant validity and convergent validity; cognitive ability / personality traits may underlie performance in AC, exercises differ in terms of psychological demands (cannot be considered parallel), too many dimensions for assessors to distinguish, exercise rather than dimensions as important parts of AC

Integrity testing

• Polygraph testing: measure physiological responses accompanying verbal responses to direct questions -> evaluate truthfulness regarding past / present behaviours; restricted only for security-related work (*not used for CWB)
• Types of questions for polygraph: non-emotional questions / emotional questions / questions about behaviour of interest; \ reactions other than guilt can trigger emotional response, some countermeasures can avoid detection
• Pen-and-paper integrity tests
• Overt - directly ask for information
• Covert / personality-based - based on assumption of common personality pattern that can be identified
• Constructs measured: conscientiousness / agreeableness / emotional stability; can test hierarchical model with 3 levels
• Incremental validity over cognitive ability, moderate predictive validity to supervisors ratings of overall performance
• Relationship of overt x personality-based tests: similar in what is being measured \ overt correlate with more honesty / supervision attitudes, personality tests correlate with impulse control etc
• False positives (shown to have high integrity) -- personality inventory will be better
• Faking - especially possible for overt tests \ do not make much difference in validity
• Legal issues: must not include questions about previous drug / alcohol use

Drug testing

• Impacts of drug use: negative association with performance, positive association with accidents / absences / job-withdrawal behaviours etc
• Issues with measurement: rely on self-report data, difficult to quantify deterioration in work performance attributable to drug use, individual differences in reactions to specific amount of drugs (metabolism / tolerance / absorption), no linear relationship between amount of drug x decline in performance
• Testing methods
• Pen-and-paper tests - may have overt questioning
• Urine tests (most often used, results returned quickly, convenient)
• Hair analysis (some drugs may leave chemical traces in hair)
• Fitness-for-duty tests, eg eye-hand coordination
• Oral fluid tests (usually very accurate)
• Accuracy of chemical tests: how much drug used? How frequently? How long ago? Level of impairment?
• Legal issues: privacy, protection of drug users, possible adverse impact