Homework 5

Homework 5#

Logistics#

Due date: The homework is due 23:59 on Monday, February 9.

You will submit your work on MarkUs. To submit your work:

  1. Download this file (Homework_5.ipynb) from JupyterHub. (See our JupyterHub Guide for detailed instructions.)

  2. Submit this file to MarkUs under the hw5 assignment. (See our MarkUs Guide for detailed instructions.) All homeworks will take place in a Jupyter notebook (like this one). When you are done, you will download this notebook and submit it to MarkUs.

Introduction#

In this homework we explore:

  • row, column selection

  • creating new columns

  • grouping

  • summary statistics

  • visualizing distributions

Question: Explore sleeping, exercising, and socializing among Canadians.

Task 1#

a) Use the pandas method read_csv to read the file pumf_education_employment.csv into a DataFrame. Store this DataFrame in a variable called pumf_data.

import pandas as pd

# Write your code here

b) Create a subset of pumf_data with only the following columns: PP_ID, HDGREE, and EMPIN. To do this follow these steps:

  • Create a list called analysis_columns with the column names.

  • Use analysis_columns to select these columns from pumf_data and store this DataFrame in a variable called pumf_data_subset.

# Write your code here

c) In the next steps you will rename the columns of pumf_data_subset according to the following table:

Old name

New name

PP_ID

Person ID

HDGREE

Highest Degree

EMPIN

Employment Income

Step 1: Create a dictionary called new_col_names with each old name as a key and each new name as the corresponding value.

# Write your code here

Step 2: Use new_col_names to rename the columns of pumf_data_subset and store the DataFrame with renamed columns in a variable called pumf_data_subset_renamed.

# Write your code here

Task 2#

Some respondents did not report a valid income (88,888,888), and others did not report positive incomes.

a) Create a boolean Series called actual_income that is True if Annual Employment Income is not equal to 88888888 and greater than zero, and False otherwise.

# Write your code here

b) Use actual_income to filter (i.e. select) the rows of pumf_data_subset_renamed where respondents had real positive incomes. Store this filtered DataFrame as a new DataFrame called income_df.

# Write your code here

c) The number of rows in a pandas DataFrame can be computed by len(). For example, len(income_df) is the number of rows in income_df. Compute the number of respondents who were removed from pumf_data_subset_renamed when it was filtered using actual_income and store this number in a variable called diff.

# Write your code here

d) Use diff to compute the percentage of respondents removed from time_use_subset_renamed_df. Round the percentage to two decimal places, and store the result value in a variable called pct_lost.

# Write your code here

Task 2#

Create a new column in income_df that converts Annual Employment Income into weekly wages. There are 52 weeks in a year, so you can calculate these weekly values by dividing the original value by 52.

Name this new column Weekly Employment Income.

# Write your code here

Task 3#

The categories for Highest Degree are not super informative unless the viewer knows which degree level each value represents. Here are the meanings of the original values in HDGREE:

Code

Description

1

No certificate, diploma or degree

2

High (secondary) school diploma

3

Non-apprenticeship trades certificate

4

Apprenticeship certificate

5

College/CEGEP/non-university certificate

6

University certificate

7

Bachelor’s degree

8

Degree above bachelor level

88

Not available

These categories are a bit too detailed, so let’s create a simplified version.

Create a new column in income_df called Education Category that contains the following categories:

  • "No Degree": No certificate, diploma or degree

  • "High School Degree": High (secondary) school diploma

  • "Certificate": “Non-apprenticeship trades certificate”, “Apprenticeship certificate”, “College/CEGEP/non-university certificate” or “University certificate”

  • "Undergraduate Degree": “Bachelor’s degree” or “Degree above bachelor level”

Using boolean conditions, identify which rows correspond to each of these four categories. Then use .loc to assign these descriptive age categories to your new Education Category column.

Recall that df.loc[row_condition, new_column_name] = value will assign value to the column new_column_name that all the rows in boolean Series row_condition that are True.

# Write your code here

b) Compute the distribution of Education Category as counts using .value_counts(), and store the count distribution in education_count. Then compute Education Category as a proportion of the total population, and store this in education_count_prop.

# Write your code here

# Write your code here

Task 5#

In this section you will explore the distributions of weekly employment income by immigration status and province.

a) Compute the mean weekly employment income by Education Level using .groupby on income_df. Include the argument as_index = False within .groupby(), so that the result prints as a DataFrame. Store this DataFrame in a variable called group_means.

# Write your code here

# Write your code here

b) Sort group_means in descending order of Weekly Employment Income. Store this sorted DataFrame in a variable called group_means_sorted

# Write your code here

c) Complete the code below to create three side-by-side boxplots from income_df using layout=(2, 2) and figsize=(20, 20) of time spent (in hours) socializing, exercising, sleeping, and their sum for each age group. Store these boxplots in a variable called boxplots.

# Write your code here

Task 6 (Written Discussion)#

a) Which group has the highest mean (average) income? Which group has the lowest? Does your ranking change if you use mean vs. median as a summary measure?

b) Which education category shows the most variability (spread) in weekly employment income? Which category shows the least? Provide a brief explanation of why these groups might show the most and least variability.

c) State one limitation of basing this data analysis on only respondents that have incomes greater than zero. Briefly explain why it’s a limitation to your findings in Tasks 4 and 5.

Answer task 6 here

Marking Rubric#

Section

0

1

2

3

Computational questions including visualizations (for each part)

auto test fails

auto test passes

NA

NA

Qualitative questions (for each part)

No answer

The question is answered but no explanation is given

The question is answered but the explanation is irrelevant or not supported

The question is answered and the explanation is supported
Contents