Overview

Workshop Goals

• Demonstrate how to work with NHTS data in R
• Generate national travel estimates with statistical confidence
• Create a publish-ready report with estimates and visuals from today’s workshop
• Preview the report we are going to create

Topics

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

“House Cleaning”

• 1 hour!?
• Write your questions down
• This presentation is reproducible
• Take it with you, download it later, go at your own pace

This presentation is intended for re-use

• Focus on the presentation itself today
  ↪ Later, follow along in RStudio at your own pace

Presentation hotkeys

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Setting Up an R Analysis Environment

Click here for latest installation instructions

Instructions provide explicit download links for

• R
• R Tools
• RStudio

Once installed, open RStudio

Setting Up an R Analysis Environment (cont.)

Make sure you have the “summarizeNHTS” R software package installed

install.packages("devtools")
devtools::install_github("Westat-Transportation/summarizeNHTS")

And now that we have installed the necessary software, load the software

library(summarizeNHTS)

OK, we’re ready!

• Your R environment for NHTS analysis is set up
• You only need to set up an environment once
• From now on, just open RStudio and load summarizeNHTS

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Accessing the Data

• Instructions for downloading the data
• Read the 2017 data into our R enviornment
• Explore the data structure and object content
• Select a subset of an NHTS data table
• Explore the codebook in R

Before we begin, let’s make sure the summarizeNHTS package is loaded.

library(summarizeNHTS)

Accessing the Data: Downloading NHTS Data

# Not Run
download_nhts_data("2001")
download_nhts_data("2009")
download_nhts_data("2017")

• You can download the 2001, 2009, and 2017 using the download_nhts_data function.
  • Downloads directly from the Oak Ridge NHTS website.
• download_nhts_data parameters
  • Dataset year
  • Dataset directory (defaults to current working directory)
• Recommend creating an RStudio Project and using the project directory.

Accessing the Data: Reading the 2017 NHTS data

nhts_data <- read_data("2017", "C:/NHTS")

• read_data is a function that reads and compiles data from CSVs.
  • Data files are packaged up (in memory) as an R object.
  • Any alterations to this object will NOT affect the CSVs.
• nhts_data is the new object we created.
  • Note: It can be named anything (within R’s syntax rules).
• Which NHTS dataset?
  • Pass '2017' to specify that we are working with the 2017 dataset.
  • Can specify directory where the data is stored (recall the download_nhts_data function).

Accessing the Data: Summarizing the data object

• The data / weights are contained within the nhts_data object we created.
  • Access these contents using $
• Use the summary function to get an overview of the data structure.

summary(nhts_data$data)

##           Length Class      Mode
## trip      60     data.table list
## person    87     data.table list
## household 59     data.table list
## vehicle   15     data.table list

• The contents of the data element include four data.table objects:
  • trip
  • person
  • household
  • vehicle
• data.tables are 2-dimensional data structures (rows X columns).
  • An extension of R’s base data.frame structure (with enhanced functionality).
• The length refers to the number of columns in each table.

Accessing the Data: Snapshot of the vehicle data

• How do I access the raw data?
• Let’s hop over to Rstudio to explore in more detail

nhts_data$data$vehicle

##          HOUSEID VEHID ANNMILES FUELTYPE HFUEL HYBRID MAKE MODEL OD_READ
##      1: 30000007    01     1000       01    -1     02   49 49032   69000
##      2: 30000007    02     2500       01    -1     02   49 49442  164000
##      3: 30000007    03      900       01    -1     02   19 19014  120000
##      4: 30000007    04      500       01    -1     02   19 19021     -88
##      5: 30000007    05    10000       01    -1     02   20 20481  300000
##     ---                                                                 
## 256111: 40794301    01     9500       01    -1     02   37 37441   60976
## 256112: 40794301    02     9600       01    -1     02   49 49032   93388
## 256113: 40794301    03     6200       01    -1     02   35 35443  143616
## 256114: 40794301    04       -1       01    -1     02   49 49471  195078
## 256115: 40794301    05     1000       01    -1     02   35 XXXXX  167141
##         VEHAGE VEHOWNED VEHOWNMO VEHTYPE VEHYEAR WHOMAIN
##      1:     10       01       -1      01    2007      03
##      2:     13       01       -1      02    2004      -8
##      3:     19       01       -1      01    1998      01
##      4:     20       01       -1      01    1997      02
##      5:     24       01       -1      04    1993      02
##     ---                                                 
## 256111:      5       01       -1      02    2012      01
## 256112:     12       02       10      01    2005      04
## 256113:     21       01       -1      02    1996      03
## 256114:     26       01       -1      04    1991      02
## 256115:     27       01       -1      01    1990      01

Accessing the Data: Subsetting

• Select specific columns

# By position
nhts_data$data$vehicle[, c(1, 3)]

# By name (single variable)
nhts_data$data$vehicle$ANNMILES

# By name
nhts_data$data$vehicle[, list(HOUSEID, ANNMILES)]

• Select specific rows

# By row numbers (first 5 rows)
nhts_data$data$vehicle[1:5, ]

# By condition
nhts_data$data$vehicle[VEHTYPE == "01", ]

# By condition (multiple values)
nhts_data$data$vehicle[VEHTYPE %in% c("01","02"), ]

• Note: Conditional subsetting will be revisited in the “Generating Estimates” secition.
• Putting it together
  • Live example: print the MAKE and MODEL of vehicles from year 2012

Accessing the Data: Codebook objects

• Access the variables and values tables for different years
  • codebook_2001, codebook_2009, codebook_2017

# 2017 variables table
head(codebook_2017$variables)

##        NAME     TABLE      TYPE
## 1: AGERANGE    person character
## 2:  AIRPORT household character
## 3:    ALT_1    person character
## 4:   ALT_16    person character
## 5:    ALT_2    person character
## 6:   ALT_23    person character
##                                                                                  LABEL
## 1:                                                                           Age Range
## 2:                                                         Influence Choice of Airport
## 3:                           Alternative Mode of Transportation: Public Transportation
## 4:                   Alternative Mode of Transportation: Public Transportation or Taxi
## 5:            Alternative Mode of Transportation: Passenger to Friend or Family Member
## 6: Alternative Mode of Transportation: Passenger to Friend/Family Member or Rental Car

# 2017 values table
head(codebook_2017$values)

##        NAME VALUE                  LABEL
## 1: AGERANGE    -8           I don't know
## 2: AGERANGE    -7 I prefer not to answer
## 3: AGERANGE    -1       Appropriate skip
## 4: AGERANGE    01          0-4 years old
## 5: AGERANGE    02         5-15 years old
## 6: AGERANGE    03        16-17 years old

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Generating Estimates

• Introduction to the summarize_data function
• Understanding summarize_data parameters
• Statistics grouped by variables
• Exploring aggregation options
• Estimates using a subset condition
• Referencing the documentation

Generating Estimates: Introduction to summarize_data

• The summarizeNHTS package has built in functions for running complex queries on the NHTS dataset
• summarize_data is the workhorse function behind these queries.

summarize_data(
  data = nhts_data,
  agg = "household_count"
)

##            W            E      S      N
## 1: 118208251 1.186759e-07 129696 129696

• What do these values mean?
  • W - Weighted statistic.
    • Count of households weighted to the population
  • E - Standard error of the weighted statistic.
    • Standard error of the weighted count of households
  • S - Surveyed/sampled statistic (unweighted statistic).
    • The count of sampled households
  • N - Number of observations/sample size.
    • The number of observations is the same as the count of sampled households in this example
• Every summarize_data query will return these fields.

Generating Estimates: Exploring summarize_data Parameters

summarize_data(
  data = nhts_data,
  agg = "household_count"
)

• Required parameters
  • data - NHTS dataset object
    • Will always be the output of read_data.
    • In our example, we stored the output in the nhts_data object.
  • agg - Aggregate function label
    • Our example used 'household_count' but agg could be a number of other labels.
• Let’s explore some of the additional parameters!
  • Categorical grouping
  • Aggregation options
  • Subset conditions
  • More…

Generating Estimates: Grouping by Variables

• What if I wanted to group these statistics by another variable?
  • Let’s use the by parameter to group by metropolitan status.

summarize_data(
  data = nhts_data,
  agg = "household_count",
  by = "IS_METRO"
)

##                   IS_METRO         W        E      S      N
## 1:       Metropolitan Area 100480095 37143.09 109250 109250
## 2: Not a Metropolitan Area  17728156 37143.09  20446  20446

• You can specify any number of variables using the by parameter.

summarize_data(
  data = nhts_data,
  agg = "household_count",
  by = c("IS_METRO","HOMEOWN")
)

##                   IS_METRO                HOMEOWN            W          E
## 1:       Metropolitan Area            I dont know     1728.364   1060.471
## 2:       Metropolitan Area I prefer not to answer    31126.577  10432.815
## 3:       Metropolitan Area                    Own 62154931.751 122801.012
## 4:       Metropolitan Area                   Rent 37328199.616 124577.033
## 5:       Metropolitan Area Some other arrangement   964108.222  87850.539
## 6: Not a Metropolitan Area I prefer not to answer     5866.829   4255.033
## 7: Not a Metropolitan Area                    Own 12363614.703 124022.494
## 8: Not a Metropolitan Area                   Rent  5135781.233 113252.628
## 9: Not a Metropolitan Area Some other arrangement   222893.706  36387.468
##        S     N
## 1:     3     3
## 2:    26    26
## 3: 81923 81923
## 4: 26568 26568
## 5:   730   730
## 6:     6     6
## 7: 16536 16536
## 8:  3700  3700
## 9:   204   204

Generating Estimates: Frequencies/Proportions

• You can use any of the following count aggregates
  • 'household_count', 'person_count', 'trip_count', 'vehicle_count'

# Person count
summarize_data(
  data = nhts_data,
  agg = "person_count"
)

##            W        E      S      N
## 1: 301599169 0.390838 264234 264234

• You can also choose to view counts as a proportion of the total using the prop parameter
  • Only useful when a by variable is specified

# Proportion of persons by WORKER, worker status
summarize_data(
  data = nhts_data,
  agg = "person_count",
  by = "WORKER",
  prop = TRUE
)

##              WORKER            W            E            S      N
## 1:  Not ascertained 6.819779e-05 2.760915e-05 6.433691e-05     17
## 2: Appropriate skip 1.508504e-01 1.263185e-03 1.057320e-01  27938
## 3:              Yes 5.205195e-01 1.690699e-03 4.855091e-01 128288
## 4:               No 3.285619e-01 2.475022e-03 4.086946e-01 107991

Generating Estimates: Numeric Aggregates

• You can use any of the following numeric aggregates
  • 'sum', 'avg', 'median'
• Must also specify a variable name using the agg_var parameter
  • Required that the variable be numeric

# Average TRPMILES, trip distance in miles
summarize_data(
  data = nhts_data,
  agg = "avg",
  agg_var = "TRPMILES"
)

##           W         E        S      N
## 1: 10.70435 0.2003875 11.45057 922916

Notes

• summarize_data handles missing value (-1,-7,-8,-9) exclusion for numeric aggregates
• The N in the example above refers to the number of trips used in the calculation

Generating Estimates: Trip Rates

• You can use either of the following trip rate aggregates
  • 'household_trip_rate' - Daily Person Trips per Household
  • 'person_trip_rate' - Daily Person Trips per Person

# Daily person Trips by worker status
summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = "WORKER"
)

##              WORKER        W          E        S      N
## 1:  Not ascertained 1.280018 1.11437619 1.647059     28
## 2: Appropriate skip 2.780250 0.03060904 2.838249  79295
## 3:              Yes 3.773681 0.01752420 3.881454 497944
## 4:               No 3.006374 0.02919963 3.206795 346305

Generating Estimates: Subsetting in summarize_data

• Pre-aggregation subset conditions can be specified using the subset parameter.
  • Argument should be passed as a string.
• Subsetting character variables

# Distribution of social/recreational trips by travel day
summarize_data(
  data = nhts_data,
  agg = "trip_count",
  by = "TRAVDAY",
  prop = TRUE,
  subset = "WHYTRP90 %in% c('07','08','10')"
)

##      TRAVDAY         W           E         S     N
## 1:    Sunday 0.1735797 0.003073281 0.1335907 33043
## 2:    Monday 0.1179881 0.002790327 0.1299642 32146
## 3:   Tuesday 0.1097795 0.001604585 0.1308981 32377
## 4: Wednesday 0.1123670 0.002679049 0.1350300 33399
## 5:  Thursday 0.1243485 0.003190797 0.1427399 35306
## 6:    Friday 0.1526481 0.003741184 0.1695486 41937
## 7:  Saturday 0.2092892 0.004953279 0.1582284 39137

• Subsetting numeric variables

# Person trip rate by Sex (for millennials)
summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = "R_SEX",
  subset = "R_AGE >= 18 & R_AGE <= 34"
)

##                     R_SEX         W          E        S     N
## 1:            I dont know 0.9505731 1.09495843 1.333333     4
## 2: I prefer not to answer 3.4089751 0.63080999 3.647059   124
## 3:                   Male 3.1543275 0.04725515 3.262248 63591
## 4:                 Female 3.4506053 0.05616977 3.590901 77427

Generating Estimates: Documentation

• Comprehensive function documentation is accessible within R!

?summarize_data

R Documentation for summarize_data

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Creating New Variables

• Derive information from other variables in the dataset
• Find new ways to to expand your analysis
• Types of derived variables include:
  • Flags (yes/no, is/is-not, has/has-not, etc)
  • Collapsing/Binning (e.g. grouping Bus and Rail into one Public Transit value)
  • (re)Categorization (manipulating categorical values to fit your needs)
  • Mathematical Calculations (variable * factor, variable / variable, etc)

Creating New Variables: Example Scenario

Example Derived Variable Coding Scenario

1. Someone’s interested in querying the NHTS for a particular travel behavior

Anthony: “I am interested in exploring how financial burden may affect travel.”
Alex: “Remember that question about walking to save money? I would include that in your analysis.”

2. Consider suggested variable’s usefulness for Anthony’s analysis:

WALK2SAVE: “I walk to places to save money.”

Values:

3. Look for potential other ways of maniupulating this variable for analysis

4. Create variable called WALK_FINANCE, a yes/no variable for the binary analysis question, “who does or does not walk to save money?”

Creating New Variables: Configuration

• A derived variable template file is included in summarizeNHTS

• Create basic and complex variables with your own logic

• Variables loaded automatically for you by read_data()

• The derived variable template file preserves details of coding for your documentation

Creating New Variables: Configuration (cont.)

Derived variable file requirements

Review File

Creating New Variables: Example 1 (Has/Has-not)

Using the Derived Variables file, create a variable with the following requirements:

• Level: Household
• Idea: Household has at least one vehicle
• Values: 1=Yes, 2=No
• Name: HAS_VEHICLE

Creating New Variables: Example 2 (Grouping)

Using the Derived Variables file, create a variable with the following requirements:

• Level: Person
• Idea: Person age categorized into four large groups
• Values: 0 to 17 = Child, 18 to 44 = Young Adult, 45 to 65 = Middle Adult, 66 and up = Older Adult
• Name: AGE_GROUP

Creating New Variables: Example 3 (Uses/Does-not-use)

Using the Derived Variables file, create a variable with the following requirements:

• Level: Person
• Idea: Person reports using a transportation network company (Uber/Lyft) in the last 30 days
• Values: 1=Yes, 2=No
• Name: USES_TNC

Creating New Variables: Example 4 (Is/Is-not)

Using the Derived Variables file, create a variable with the following requirements:

• Level: Household
• Idea: Is this household in a Metropolitan Statistical Area?
• Values: 1=Yes, 2=No
• Name: IS_METRO

Creating New Variables: Summary

• Include new ways of grouping your statistic
• Variables become available on next data read
• This is a preferred organization method, but not necessary
  • Alternative: attach them to the appropriate csv file yourself
  • Alternative: compute and attach to the appropriate table level in R after read_data()

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Visualizing Estimates

• Demonstrate the 3 core visualization functions
  • make_table - Create report-ready, formatted tables.
  • make_chart - Create interactive bar charts.
  • make_map - Create interactive choropleth maps.
• Quick and easy steps for visuals:
  1. Assign the output of summarize_data to a new object.
  2. Feed that object to one of the visualization functions to output the results.
• Numeric formatting supported for all 3 viz functions.

Visualizing Estimates: Tables (Introductory)

statistic <- summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = "WORKER"
)

make_table(statistic)

Visualizing Estimates: Tables (Advanced)

• Example of a 3-way table with custom configuration

statistic <- summarize_data(
  data = nhts_data,
  agg = "person_count",
  by = c("TRAVDAY","OCCAT","EDUC"),
  exclude_missing = TRUE
)

make_table(
  tbl = statistic,
  title = "Table 1: Distribution of Persons (%) by Travel Day, Job Category, and Educational Attainment",
  output = c(W = "Weighted Percentage", N = "Sample Size"),
  row_vars = c("EDUC","OCCAT")
)

Visualizing Estimates: Charts (Introductory)

statistic <- summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = "WORKER",
  exclude_missing = TRUE
)

make_chart(statistic)

Visualizing Estimates: Charts (Advanced)

Person Trip Rate by Sex, Worker Status, and Travel Day of Week

statistic <- summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = c("R_SEX","WORKER","TRAVDAY"),
  exclude_missing = TRUE
)

# Specify fill and facet
make_chart(
  tbl = statistic, 
  fill = "WORKER",
  facet = "TRAVDAY",
  palette = "Accent"
)

Visualizing Estimates: Maps (Introductory)

statistic <- summarize_data(
  data = nhts_data,
  agg = "person_count",
  by = "CENSUS_D"
)

make_map(statistic)

Visualizing Estimates: Maps - Built in Geography Layers

• Census Regions
  • Variable: CENSUS_R
  • Layer: census_region_layer
• Census Divisions
  • Variable: CENSUS_D
  • Layer: census_division_layer
• States
  • Variable: HHSTFIPS
  • Layer: state_layer / state_tile_layer
• CBSA
  • Variable: HH_CBSA
  • Layer: cbsa_layer

Visualizing Estimates: Maps (Advanced)

• We can even embed charts in map geographies!

Include a second table grouping by the original geography plus one variable.

statistic1 <- summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = "HHSTFIPS",
  exclude_missing = TRUE
)

statistic2 <- summarize_data(
  data = nhts_data,
  agg = "person_trip_rate",
  by = c("HHSTFIPS","WORKER"),
  exclude_missing = TRUE
)

map <- make_map(
  tbl = statistic1, 
  tbl2 = statistic2
)

map

Formatting

• All 3 visualization function support the same value formatting options:
  • digits - Number of decimal places to use
  • percentage - Treat proportions as percentages
    • TRUE or FALSE
  • scientific - Use scientific notation
    • TRUE or FALSE
  • multiplier - A value multiplier
    • Ex: To display a value “In Thousands”, use multiplier = 1000
• Formatting example with make_table

statistic <- summarize_data(
  data = nhts_data,
  agg = "trip_count",
  by = "PRMACT",
  exclude_missing = TRUE
)

make_table(
  tbl = statistic,
  title = "Trip Count by Primary Activity (in Millions)",
  output = c(W = "Trip Count (Millions)", E = "SE"),
  digits = 0,
  multiplier = 1000000
)

Next Topic

1. Setting Up an R Analysis Environment
2. Accessing the Data
3. Generating Estimates
4. Creating New Variables
5. Visualizing Estimates
6. Producing a Travel Analysis Report

Producing a Travel Analysis Report

• Put your analysis and results directly into a shareable document
• Use simple Markdown syntax in RStudio to create word, pdf, and html reports
• Use RStudio and today“s examples to produce an NHTS data based report right now
• Start with a template file to jump off from
  • Template R Markdown file