06 Paired and Indepdendent Samples t Tests

Oct. 15, 2020

Today

Reproducing results from “The Sound of Intellect” Experiment 1 and Experiment 3a.

Learning Outcomes (Paired Samples t)

Do the following for paired samples t tests:

  1. Conduct power/sensitivity analyses.
  2. Conduct paired samples t tests in R.
  3. Evaluate assumption of normality.
  4. Produce appropriate visualizations of paired data.
  5. Report results in APA style.
  6. Evaluate the strength of evidence they provide.

Paired Samples t

What is a Paired Samples t?

A paired samples t test is used to compare two means that were sampled from the same set of participants.

Seen in:

  • Pre-to-post differences.
  • Crossed designs.

Paired Samples t = One-Sample t

Statistically, a paired samples t test is just a one-sample t test on the difference scores.

Difference scores (AKA change scores):

\(\Delta{X} = X_2 - X_1\)

Was the average change greater than/less than/different from zero?

Example: The Sound of Intellect

Study Aims

What cues do people use to infer intellect? Differences between reading, hearing, and watching (and hearing) a job candidate’s pitch.

Research Questions

Two research questions from experiment 1 that we’re looking at.

RQ1

Do job candidates think their written pitch will be perceived more or less positively than their spoken pitch?

RQ2

Do job candidates expect their chances of being hired to be different for their written and spoken pitches?

Why does this Matter?

“Theoretically, such expectations matter because they indicate whether the cues that convey mental capacities in social interaction are obvious to those in the midst of the interaction. Practically, such expectations matter because they could guide how candidates approach potential employers. Candidates who believe their spoken pitch will be judged exactly the same as their written pitch may see no reason to seek voice time with a potential employer.”

Hypotheses

The (implied) hypotheses are…

Hypothesis 1

Candidates will predict written and spoken pitches will be perceived differently.

Hypothesis 2

Candidates will predict that employers’ interest will vary based on whether they observed the written or spoken pitches.

Basic Study Design

Record Pitch

Photo by CoWomen on Unsplash

Write Pitch

Photo by Hannah Olinger on Unsplash

Survey

Survey Questions

Experiment 1 survey

Analytic Plan

Conduct a paired samples t test comparing participants’ predicted positivity ratings for their spoken and written pitches.

Open RStudio

Install/Load Packages

These are the packages we’ll be using in the lab today.

library(readxl)
library(psych)
library(pwr)
library(tidyr)
library(ggplot2)
library(foreign)

Power/Sensitivity Analysis

Review: Power vs. Sensitivity Analysis

  • When/why would you conduct a power analysis?
  • When/why would you conduct a sensitivity analysis?

Underpowered

“…these predictions were underpowered given the sample size of only 18 candidates…” (p. 880)

How underpowered? How are we defining underpowered?

Sensitivity Analysis 50% Power

What is the smallest population effect 50% of samples of N = 18 would detect?

pwr::pwr.t.test(n = 18,
  d = NULL,
  sig.level = 0.05,
  power = .50,
  type = "paired",
  alternative = "two.sided"
)
## 
##      Paired t test power calculation 
## 
##               n = 18
##               d = 0.4897239
##       sig.level = 0.05
##           power = 0.5
##     alternative = two.sided
## 
## NOTE: n is number of *pairs*

Sensitivity Analysis 80% Power

What is the smallest population effect 80% of samples of N = 18 would detect?

pwr::pwr.t.test(n = 18,
  d = NULL,
  sig.level = 0.05,
  power = .80,
  type = "paired",
  alternative = "two.sided"
)
## 
##      Paired t test power calculation 
## 
##               n = 18
##               d = 0.7007201
##       sig.level = 0.05
##           power = 0.8
##     alternative = two.sided
## 
## NOTE: n is number of *pairs*

Sensitivity Analysis 95% Power

What is the smallest population effect 95% of samples of N = 18 would detect?

pwr::pwr.t.test(n = 18,
  d = NULL,
  sig.level = 0.05,
  power = .95,
  type = "paired",
  alternative = "two.sided"
)
## 
##      Paired t test power calculation 
## 
##               n = 18
##               d = 0.902463
##       sig.level = 0.05
##           power = 0.95
##     alternative = two.sided
## 
## NOTE: n is number of *pairs*

Explore Data

tRy it! Data Import

Use readxl::read_excel() to import the Excel file.

dta <- readxl::read_excel(
  path = "../data/Job candidate predictions.xlsx",
  sheet = 1
)

Inspect the Data Frame

dta
## # A tibble: 18 x 10
##     `P#` Company PosWrit HireWrit PosSpoke HireSpoke `Times given`   Age Gender
##    <dbl> <chr>     <dbl>    <dbl>    <dbl>     <dbl> <chr>         <dbl> <chr> 
##  1     1 Google        3        3        4         4 3 to 5           26 M     
##  2     2 BCG           4        4        3         3 0                27 M     
##  3     3 Sprint        4        4        4         4 2                31 F     
##  4     4 Micros…       4        4        3         2 3                29 F     
##  5     5 Kleine…       3        3        4         3 3                26 M     
##  6     6 Raymon…       5        5        4         4 2-3 times, n…    29 M     
##  7     7 McKins…       3        3        3         3 2 for job in…    28 F     
##  8     8 Wilson…       4        3        4         4 0                28 M     
##  9     9 Samsun…       4        4        3         3 0                32 M     
## 10    10 Kraft …       1        1        5         4 1                28 M     
## 11    11 Gates …       3        1        4         4 0                24 F     
## 12    12 Spotify       2        2        4         3 0                28 M     
## 13    13 Mattel        3        3        4         4 0                28 F     
## 14    14 Coca C…       3        3        5         5 0                28 F     
## 15    15 Accent…       3        3        2         2 1                30 M     
## 16    16 MetLife       2        2        3         3 0                27 M     
## 17    17 McKins…       3        2        3         2 2                32 F     
## 18    18 Kaiser…       4        4        3         2 2 to 3           27 M     
## # … with 1 more variable: Ethnicity <chr>

Structure of the Data Frame

str(dta)
## tibble [18 × 10] (S3: tbl_df/tbl/data.frame)
##  $ P#         : num [1:18] 1 2 3 4 5 6 7 8 9 10 ...
##  $ Company    : chr [1:18] "Google" "BCG" "Sprint" "Microsoft" ...
##  $ PosWrit    : num [1:18] 3 4 4 4 3 5 3 4 4 1 ...
##  $ HireWrit   : num [1:18] 3 4 4 4 3 5 3 3 4 1 ...
##  $ PosSpoke   : num [1:18] 4 3 4 3 4 4 3 4 3 5 ...
##  $ HireSpoke  : num [1:18] 4 3 4 2 3 4 3 4 3 4 ...
##  $ Times given: chr [1:18] "3 to 5" "0" "2" "3" ...
##  $ Age        : num [1:18] 26 27 31 29 26 29 28 28 32 28 ...
##  $ Gender     : chr [1:18] "M" "M" "F" "F" ...
##  $ Ethnicity  : chr [1:18] "Asian American" "White European" "Indian-American (Sub-continent)" "Indian" ...

Summary of the Data Frame

summary(dta)
##        P#          Company             PosWrit         HireWrit   
##  Min.   : 1.00   Length:18          Min.   :1.000   Min.   :1.00  
##  1st Qu.: 5.25   Class :character   1st Qu.:3.000   1st Qu.:2.25  
##  Median : 9.50   Mode  :character   Median :3.000   Median :3.00  
##  Mean   : 9.50                      Mean   :3.222   Mean   :3.00  
##  3rd Qu.:13.75                      3rd Qu.:4.000   3rd Qu.:4.00  
##  Max.   :18.00                      Max.   :5.000   Max.   :5.00  
##     PosSpoke       HireSpoke     Times given             Age       
##  Min.   :2.000   Min.   :2.000   Length:18          Min.   :24.00  
##  1st Qu.:3.000   1st Qu.:3.000   Class :character   1st Qu.:27.00  
##  Median :4.000   Median :3.000   Mode  :character   Median :28.00  
##  Mean   :3.611   Mean   :3.278                      Mean   :28.22  
##  3rd Qu.:4.000   3rd Qu.:4.000                      3rd Qu.:29.00  
##  Max.   :5.000   Max.   :5.000                      Max.   :32.00  
##     Gender           Ethnicity        
##  Length:18          Length:18         
##  Class :character   Class :character  
##  Mode  :character   Mode  :character  
##                                       
##                                       
##

tRy it! M and SD

Compute means and standard deviations of how positively participants expected to be evaluated.

mean(dta$PosSpoke)
sd(dta$PosSpoke)

mean(dta$PosWrit)
sd(dta$PosWrit)

Using psych::describe()

This is a handy function that will describe columns in your data frame.

psych::describe(dta[, c("PosSpoke", "PosWrit")])
##          vars  n mean   sd median trimmed  mad min max range  skew kurtosis
## PosSpoke    1 18 3.61 0.78      4    3.62 1.48   2   5     3  0.01    -0.67
## PosWrit     2 18 3.22 0.94      3    3.25 1.48   1   5     4 -0.42    -0.11
##            se
## PosSpoke 0.18
## PosWrit  0.22

Conduct Tests

Job Candidates’ Predictions (Results)

Together

These participants did not predict that they would be evaluated differently when employers listened to their spoken pitches (M = 3.61, SD = 0.78) than when employers read their written pitches (M = 3.22, SD = 0.94), paired t(17) = 1.20, p = .25, d = 0.45.

For Practice

“They also did not expect any difference in their likelihood of getting hired depending on whether employers listened to their spoken pitches (M = 3.28, SD = 0.89) or read their written pitches (M = 3.00, SD = 1.08), paired t(17) = 0.80, p = .44, d = 0.29.”

Reproducing Findings

tRy it! Difference Scores

Add a column to your data frame that contains the difference between expected evaluation of spoken and written pitches.

dta$PosDiff <- dta$PosSpoke - dta$PosWrit

tRy it! Paired t Test (Option 1)

There are two ways to do a paired-samples t test. The first is to do a one-sample t test of the difference scores.

Conduct a one sample t test against the null hypothesis that the mean of the difference scores is 0.

Paired t Test (Option 1)

result1 <- t.test(x = dta$PosDiff)

result1
## 
##  One Sample t-test
## 
## data:  dta$PosDiff
## t = 1.1974, df = 17, p-value = 0.2476
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.2963399  1.0741177
## sample estimates:
## mean of x 
## 0.3888889

tRy it! Paired t Test (Option 2)

Try the second way of getting the same result, which is using t.test() with the argument paired = TRUE.

Paired t Test (Option 2)

result2 <- t.test(x = dta$PosSpoke, y = dta$PosWrit, paired = TRUE)

result2
## 
##  Paired t-test
## 
## data:  dta$PosSpoke and dta$PosWrit
## t = 1.1974, df = 17, p-value = 0.2476
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.2963399  1.0741177
## sample estimates:
## mean of the differences 
##               0.3888889

Compare the Results

## 
##  One Sample t-test
## 
## data:  dta$PosDiff
## t = 1.1974, df = 17, p-value = 0.2476
## alternative hypothesis: true mean is not equal to 0
## 95 percent confidence interval:
##  -0.2963399  1.0741177
## sample estimates:
## mean of x 
## 0.3888889
## 
##  Paired t-test
## 
## data:  dta$PosSpoke and dta$PosWrit
## t = 1.1974, df = 17, p-value = 0.2476
## alternative hypothesis: true difference in means is not equal to 0
## 95 percent confidence interval:
##  -0.2963399  1.0741177
## sample estimates:
## mean of the differences 
##               0.3888889

Assumptions of Paired t

Assumptions of Paired t

The assumptions are the same as for a one-sample t, but they are assumptions about the difference scores.

  1. Difference scores come from a normally distributed population.
  2. Observations (i.e., difference scores) are independent.

Evaluating Assumptions

We can assess how tenable the normality assumption is in the same ways we did for the one-sample t test:

  1. Shapiro–Wilk Test.
  2. Visual inspection of histogram and q–q plot.
  3. Skew and kurtosis values.

Shapiro–Wilk Test

shapiro.test(dta$PosDiff)
## 
##  Shapiro-Wilk normality test
## 
## data:  dta$PosDiff
## W = 0.86361, p-value = 0.01398

Histogram

hist(dta$PosDiff)

Histogram

ggplot2 Histogram

ggplot(dta, aes(PosDiff)) +
  geom_histogram(binwidth = 1)

ggplot2 Histogram

Q–Q Plot

qqnorm(dta$PosDiff, ylim = c(-2, 4))
qqline(dta$PosDiff)

Q–Q Plot