Observational Measurement of Behavior

ISBN-13: 9781681252469 / Angielski / Miękka / 2018 / 296 str.

Paul J. Yoder; Frank J. Symons; Blair Lloyd

Opis

Introduces graduate students to observational measurement of behaviour. It discusses in-depth the theoretical considerations underlying observational research and provides specific recommendations for effective techniques and practices.

Spis treści

• 1: Introduction and Measurement Contexts
• Overview
• Definition of Systematic Observation Using Count Coding
• Rationale for Systematic Observation Using Count Coding
• Importance of Falsifiable Hypotheses
• The Continuum of State-Likeness to Trait-Likeness
• Context-Dependent Behavior
• Person Characteristics
• Generalized Behavioral Tendencies
• Skills
• The Relative Scientific Value of Different Objects of Measurement
• Ecological Validity and Representativeness
• Conclusions and Recommendations
• References
• 2: Validation of Observational Variables
• Overview
• The Changing Concept of Validation
• Understanding Which Types of Validation Evidence Are Most Relevant for Different Research Designs, Objects of Measurement, and Research Purposes
• Content Validation
• Definition of Content Validation
• Different Traditions Vary on the Levels of Importance Placed on Content Validation
• Weaknesses of Content Validation
• Sensitivity to Change
• Definition of Sensitivity to Change
• Influences on Sensitivity to Change
• Weaknesses of Sensitivity to Change
• Treatment Utility
• Definition of Treatment Utility
• Weaknesses of Treatment Utility
• Criterion-Related Validation
• Definition of Criterion-Related Validation
• Primary Appeal of Criterion-Related Validation
• Weaknesses of Criterion-Related Validation
• Construct Validation
• Definition of Construct Validation
• Discriminative Validation
• Nomological Validation
• Multitrait, Multimethod Validation
• An Implicit “Weakness” of Science?
• Recommendations
• References
• 3: Measuring Person Characteristics
• Overview
• Contextual Measurement Error
• Definition of Measurement Context
• A Brief Overview of Measurement Theory
• Definition of Contextual Measurement Error
• Representativeness
• Contextual Measurement Error in Measures Of Generalized Behavioral Tendencies
• Averaging Scores Across Contexts Improves Measures of Generalized Behavioral Tendencies
• Aggregates Tend to Improve Estimates of Known True Score.
• Aggregates Tend to Improve Construct Validity.
• Aggregates Tend to Improve Stability.
• Controlling Influential Contextual Variables Stabilizes Observed Scores for Highly Variable Person Characteristics
• Why Naturalistic Observations Are Not Necessarily More Representative Than Contrived Ones
• Why Skills Are Often Measured in Structured Measurement Contexts
• Why Skills Are Often Assessed in Clinics or Labs
• The Link Between Stability and Construct Validity
• Recommendations and Conclusions
• References
• 4: Designing or Adapting Coding Manuals
• Overview
• Definition of a Coding Manual
• Deciding Whether to Use an Existing Coding Manual or to Construct a New One
• Recommended Steps for Modifying or Designing Coding Manuals
• Defining When to Start and Stop Coding
• Conceptually Defining the Context-Dependent Behavior or the Generalized Characteristic
• Defining the Highest Level of Codable Behavior
• Determining the Level of Distinction Coders Have to Make
• Organizing the Coded Categories into Mutually-Exclusive Sets
• Physically Based Definitions, Socially Based Definitions, or Both?
• Defining the Lowest Level Categories
• Source of Conceptual and Operational Definitions
• A Qualitative Approach to Identifying Definitions
• Defining Segmenting Rules
• The Potential Value of Flowcharts
• Do Coding Manuals Need to be Sufficiently Short to be Included in Methods Sections?
• Recommendations and Conclusions
• References
• 5: Coding
• Overview
• The Elements of an Observational Measurement System
• Behavior Sampling
• The Superordinate Distinctions: Continuous versus Intermittent
• The Subordinate Distinctions: Timed-Event versus Event versus Interval
• Timed-Event Sampling
• Event Sampling
• Interval Sampling
• Types of Interval Sampling
• Whole Interval Sampling
• Momentary Interval Sampling
• Partial Interval Sampling
• The Importance of Knowing What Metric the Investigator Wants to Estimate
• Summary of Behavior Sampling
• Participant Sampling
• Focal Sampling
• Multiple Pass Sampling
• Conspicuous Sampling
• Reactivity
• Live Coding versus Recording the Observation For Later Coding
• Live Coding
• Coding from Recorded Sessions
• Recording Coding Decisions
• Recommendations and Conclusions
• References
• 6: Common Metrics of Observational Variables
• Overview
• Definition of Metric
• Quantifiable Dimensions of Behavior
• Proportion Metrics
• Proportion Metrics Change the Meaning of Observational Variables
• Scrutinizing Proportions
• An Implicit Assumption of Proportion Metrics
• Testing Whether the Data Fit the Assumption of Proportion Metrics
• Consequences of Using a Proportion When the Data Do Not Fit the Assumption
• Alternative Methods to Control Influential Contextual Variables
• Statistical Control
• Procedural Control
• Aggregate Measures of Generalized Person Characteristics
• Weighted Count
• Unit-Weighted Aggregates
• Group Analysis of Observational Variables
• Transforming the Metric
• Bootstrapping
• Recommendations and Conclusions
• References
• 7: Observer Training and Preventing Observer Drift
• Overview
• Point-by-point Agreement and Disagreement
• Point-by-point Agreement of Interval Sampled Data
• Point-by-point Agreement of Timed-Event Data
• Discrepancy Matrices
• Discrepancy Discussions
• Using Discrepancy Discussions to Train Observers
• Creating Criterion-coding Standards.
• Remaining Steps to Train Observers
• Preventing Observer Drift
• Method of Selecting Sessions for Agreement Checks
• Remaining Steps to Preventing or Addressing Observer Drift
• Recommendations
• References
• 8: Interobserver Agreement and Reliability of Observational Variables
• Overview
• Additional Purposes of Point-by-Point Agreement
• Added Principles When Agreement Checks Are Used to Estimate Interobserver “Reliability” of Observational Variable Scores
• Exhaustive Coding Spaces Revisited
• The Effect of Chance on Agreement
• Common Indices of Point-by-Point Agreement
• Occurrence Percentage Agreement
• Nonoccurrence Percentage Agreement
• Total Percentage Agreement
• Kappa
• Base Rate and Chance Agreement Revisited
• Intraclass Correlation Coefficient (ICC) as an Index of Interobserver Reliability in Group Designs
• Options for Running ICC With SPSS
• Between-Participant Variance on the Variable of Interest Affects ICC
• Using ICC as a Measure of Interobserver Reliability for Predictors and Dependent Variables in Group Designs
• The Interpretation of SPSS Output for ICC
• The Conceptual Relation Between Interobserver Agreement and ICC
• Consequences of Low or Unknown Interobserver Reliability
• Recommendations
• References
• 9: Introduction to Sequential Analysis
• Overview
• Definition of Terms Used in this Chapter
• Sequential versus Nonsequential Variables
• Sequential Associations are not Sufficient Evidence for Causal Inferences
• Coded Units and Exhaustiveness
• Contingency Tables
• Three Major Types of Sequential Analysis
• Event Lag
• Concurrent Interval
• Event Lag with Pauses (to replace time window method)
• Explanation for no longer focusing on time window
• Indices of Sequential Association: Controlling for Chance
• Existing Indices of Sequential Association: Advantages and Disadvantages
• Transitional/conditional probabilities
• Yule’s Q
• Risk Difference/Operant Contingency Value
• Other commonly-used indices and why we do not focus on them (e.g., z)
• Recommendations and Conclusions
• References
• 10: Identifying and Addressing Research Questions Involving Sequential Associations
• Overview
• Sequential Analysis in Group Designs
• Types of Research Questions and Methods to Address Them
• Indices of Sequential Association as Dependent Variables
• Testing the Significance of a Mean Sequential Association
• Testing the Between-Group Difference in Mean Sequential Associations
• Testing the Within-Subject Difference in Sequential Associations
• Sequential Analysis in Single-Case Designs
• Types of Research Questions
• Descriptive Questions Involving Behavior-Environment Associations to Inform Experimental Analyses
• Descriptive Questions to Inform Temporal Distribution of One or More Target Behaviors
• Indices of Sequential Association as Dependent Variables in Single-Case Designs
• Indices of Sequential Association to Inform Procedural Fidelity
• Methods to Address Sequential Analysis Research Questions in a Single Case Framework
• Why Significance Testing is Inappropriate at the Level of the Individual Participant
• More on the Term ‘Operant Contingency’
• Summary of Analysis Methods Used in Behavior Analytic Literature
• Conditional Probabilities
• Lag Sequential Analysis
• Contingency Space Analysis
• What is “Enough Data” and How Do We Attain It?
• Proposed Solutions for Insufficient Data
• Summary and Recommendations
• References
• 11: Generalizability Theory
• Overview
• Scope of This Chapter
• Overview of G Theory and Definition of Terms
• An Example Observer by Context G and D Study
• Rationale for Preferring the Absolute G Coefficient
• Example Applications of D Studies
• An Ongoing Controversy
• Recommendations and Conclusions
• References
• 12: Best Practices in Observational Measurement
• Glossary
• Index