Activity: Grading the professor

Simple and multiple linear regression

Many college courses conclude by giving students the opportunity to evaluate the course and the instructor anonymously. However, the use of these student evaluations as an indicator of course quality and teaching effectiveness is often criticized because these measures may reflect the influence of non-teaching related characteristics, such as the physical appearance of the instructor. The article titled, “Beauty in the classroom: instructors’ pulchritude and putative pedagogical productivity” (Hamermesh and Parker, 2005) found that instructors who are viewed to be better looking receive higher instructional ratings. (Daniel S. Hamermesh, Amy Parker, Beauty in the classroom: instructors pulchritude and putative pedagogical productivity, Economics of Education Review, Volume 24, Issue 4, August 2005, Pages 369-376, ISSN 0272-7757, 10.1016/j.econedurev.2004.07.013. http://www.sciencedirect.com/science/article/pii/S0272775704001165.)

In this activity you will analyze the data from this study in order to learn what goes into a positive professor evaluation.

The data were gathered from end of semester student evaluations for a large sample of professors from the University of Texas at Austin. In addition, six students rated the professors’ physical appearance. (This is a slightly modified version of the original data set that was released as part of the replication data for Data Analysis Using Regression and Multilevel/Hierarchical Models (Gelman and Hill, 2007).) The result is a data frame where each row contains a different course and columns represent variables about the courses and professors.

Learning goals

• Fitting a linear regression with a single numerical and categorical predictor
• Interpreting regression output in context of the data
• Comparing models

Getting started

Note

Download the activity Quarto file here and place it in your class activities folder.

• Open the R Quarto document.

• Render the document to make sure it compiles without errors.

Update the YAML (header) of your Quarto file with your information and render.

Tip

What to do if you face an error when installing and loading the tidymodels package.

• The newest version of tidymodels requires you to either:

  • Update the rlang package. Use install.packages("rlang") to update.

  • Update your version of RStudio. To do so, you will need to download the newest release of RStudio (that came out just after we started the semester):

    • https://posit.co/download/rstudio-desktop/#download

    • Note: you do not need to uninstall anything in order to update. Simply download the newest version and install.

Packages

We’ll use the tidyverse package for much of the data wrangling and visualization, the tidymodels package for modeling and inference, and the data lives in the dsbox package.

library(tidyverse) 
library(tidymodels)
library(openintro)

Data

The data can be found in the openintro package, and it’s called evals.

Since the dataset is distributed with the package, we don’t need to load it separately; it becomes available to us when we load the package. You can find out more about the dataset by inspecting its documentation, which you can access by running ?evals in the Console or using the Help menu in RStudio to search for evals. You can also find this information here.

Part 1: Explore the data

Exploratory Data Analysis

1. Visualize the distribution of score. Is the distribution skewed? What does that tell you about how students rate courses? Is this what you expected to see? Why, or why not? Include any summary statistics and visualizations you use in your response.

evals %>% 
  ggplot(aes(x = score))+
  geom_histogram()

2. Visualize and describe the relationship between score and bty_avg.

evals %>% 
  ggplot(aes(x = bty_avg, y = score))+
  geom_point()

3. Recreate the plot from Step 2, but this time use geom_jitter(). What does “jitter” mean? What was misleading about the initial plot?

evals %>% 
  ggplot(aes(x = bty_avg, y = score))+
  geom_jitter()

Part 2 Linear regression with a numerical independent variable

Linear model is in the form \(\hat{y} = b_0 + b_1 x\).

4. Let’s see if the apparent trend in the plot is something more than natural variation. Fit a linear model called score_bty_fit to predict average professor evaluation score by average beauty rating (bty_avg). Based on the regression output, write the linear model (as a comment inside the code chunk).

# Run model
score_bty_fit <- lm(score ~ bty_avg, data = evals)

# See model results
summary(score_bty_fit)

5. Recreate the plot from Step 2, and add the regression line to this plot in orange color, with shading for the uncertainty of the line turned off.

evals %>% 
  ggplot(aes(x = bty_avg, y = score))+
  geom_point() +
  geom_smooth(method="lm", color = "orange", se = FALSE)

6. Interpret the slope of the linear model in context of the data. Write your response below.

# score = 3.9 + 0.067*bty_avg

#On average, a one unit increase in the average beauty rating bty_avg is associated with an increase in the professor evaluation score of 0.067.

7. Interpret the intercept of the linear model in context of the data. Comment on whether or not the intercept makes sense in this context. Write your response below.

With a beauty rating of 0, on average, the professor evaluation score is estimated to be 3.9.

8. Determine the \(R^2\) of the model and interpret it in context of the data. Write your response below.

# Adjusted R-squared: 0.03293

# The proportion of variation in score (our dependent variable) explained by the model is 0.03. In other words, 3% of the variation in professor score is explained by the average beauty rating bty_avg.

9. Working with residuals. Calculate the required variables and make a scatter plot which shows the residuals \(e_i = y_i - \widehat{y_i}\) on the y-axis and the predicted values \(\widehat{y_i}\) on the x-axis.

evals_with_residuals <- 
  evals %>% 
  mutate(estimated_score = 0.067 * bty_avg + 3.9, # Model equation for estimated dependent variable
         residual = score - estimated_score)

evals_with_residuals %>%
  ggplot(aes(x = estimated_score, y = residual)) +
  geom_point() +
  geom_line(y = 0, color = "red") 

What aspects of the residual plot show that the linear model is appropriate or not? Are there any aspects of the residual plot that cause you to be concerned about the fitting of the linear model? Write your response.

Part 3 Linear regression with a categorical independent variable

10. Fit a new linear model called score_gender_fit to predict average professor evaluation score based on gender of the professor. Based on the regression output, write the linear model and interpret the slope and intercept in context of the data.

score_gender_fit <- lm(score ~ gender, data = evals)

summary(score_gender_fit) 

# score = 4.1 + 0.14*gendermale

11. What is the equation of the line corresponding to male professors? What is it for female professors? Write your response below.

# For males, the parameter estimate is multiplied by 1. The variable gender is 0 if female and 1 if male.

# Male: 
## score = 4.1 + 0.14*gendermale
### score = 4.1 + 0.14*1 = 4.24

# Female
## score = 4.1 + 0.14*0 = 4.1

Part 4 Multiple linear regression

Simple linear regression

1. (Refresher from above) Fit a linear model (one you have fit before): score_bty_fit, predicting average professor evaluation score based on average beauty rating (bty_avg) only. Write the linear model, and note the adjusted \(R^2\).

# Your code here

Multiple linear regression

2. Fit a linear model: score_bty_gen_fit, predicting average professor evaluation score based on average beauty rating (bty_avg) and gender. Write the linear model, and note the \(R^2\) and the adjusted \(R^2\).

score_bty_gen_fit <- lm(score ~ bty_avg + gender, data = evals)
summary(score_bty_gen_fit)

# score = 3.75 + 0.074*bty_avg + 0.17*gendermale
# Adjusted R-squared:  0.05503

3. Interpret the slopes and intercept of score_bty_gen_fit in context of the data.

# Slopes:

## A one unit change in average beauty rating bty_avg is associated, on average, with a 0.074 increase in professor evaluation score.

## Professors who are men, on average, are expected to have an evaluation score 0.074 greater than women.

# Intercept

## Professors who are women are expected, on average, to have an evaluation score of 3.75.

4. What percent of the variability in score is explained by the model score_bty_gen_fit?

# The proportion of variation in score explained by the model is 0.055. In other words, 5.5% of the variation in professor score is explained by the average beauty rating and gender of the professor.

More practice (not required)

Categorical independent variable

1. Fit a new linear model called score_rank_fit to predict average professor evaluation score based on rank of the professor. Based on the regression output, write the linear model and interpret the slopes and intercept in context of the data.

score_rank_fit <- lm(score ~ rank, data = evals)

summary(score_rank_fit)

# score = 4.28 - 0.13*ranktenure_track - 0.145*ranktenured

# Adjusted R-squared: 0.007332

# The proportion of variation in score (our dependent variable) explained by the model is 0.007. In other words, 0.7% of the variation in professor score is explained by a professor's rank. 

2. Create a new variable called rank_relevel where "tenure track" is the baseline level. Use the fct_relevel function to specify the baseline (first) level. See our notes on factors.

evals <- evals %>% 
  mutate(rank_relevel = fct_relevel(
    rank,
    "tenure track", "tenured","teaching"
  ))

# distinct(evals, rank) # Run this code to see the categories in rank variable

3. Fit a new linear model called score_rank_relevel_fit to predict average professor evaluation score based on rank_relevel of the professor. This is the new (releveled) variable you created in the previous step. Based on the regression output, write the linear model and interpret the slopes and intercept in context of the data. Also determine and interpret the \(R^2\) of the model.

# Your code here

Practice data wrangling and categorical independent variables

4. Create another new variable called tenure_eligible that labels "teaching" faculty as "no" and labels "tenure track" and "tenured" faculty as "yes".

# Your code here

5. Fit a new linear model called score_tenure_eligible_fit to predict average professor evaluation score based on tenure_eligibleness of the professor. This is the new (regrouped) variable you created in the previous step. Based on the regression output, write the linear model and interpret the slopes and intercept in context of the data. Also determine and interpret the \(R^2\) of the model.

# Your code here

6. What is the equation of the line corresponding to just male professors from Part 4 question 2 above?

7. For two professors who received the same beauty rating, which gender tends to have the higher course evaluation score?

8. How does the relationship between beauty and evaluation score vary between male and female professors?

# Your code here

9. How do the adjusted \(R^2\) values of score_bty_gen_fit and score_bty_fit compare? What does this tell us about how useful gender is in explaining the variability in evaluation scores when we already have information on the beauty score of the professor.

10. Compare the slopes of bty_avg under the two models (score_bty_fit and score_bty_gen_fit). Has the addition of gender to the model changed the parameter estimate (slope) for bty_avg?

11. Create a new model called score_bty_rank_fit with gender removed and rank added in. Write the equation of the linear model and interpret the slopes and intercept in context of the data.

# Your code here