Alternative parsers in package space

Jim Hester

DSC 2017 - Syntax Extensions for R

"C:\\foo\\bar"

r"C:\foo\bar"

glue("pi = {pi}")

g"pi = {pi}"

mtcars %>%
  subset(hp > mean(hp)) %>%
  head()

iris |>
  subset(p > mean(hp)) |>
  head()

list(1, 2, 3, list(4, 5, 6))

[1, 2, 3 [4, 5, 6]]

Changing base R parser is challenging

• build consensus
• avoid conflicts / grammar ambiguities
• requires extensive testing

altparsers package

• https://github.com/jimhester/altparsers
• Multiple parsers available
  • User / package author extendable
• Can mix parsers in same package
• REPL(s) available for interactive use
• Experiment in package space

tidy parser

• No plans to use widely
• Based on R bison parser
• Many features from previous talk
  • raw strings:

    grepl(r"\w+")

  • glue strings:

    x <- 1; g"x = {x}"

  • native pipes:

    mtcars |> filter(hp > 150) |> select(hp, mpg)

  • list generation:

    [1, 2, 3 [5, 5, 6]]

Unquoting operators (!! and !!!)

# Normal R parser
`!`(`!`(foo))

# Would prefer
`!!`(foo)

# Prefer tighter binding
!!x + y

• 2k+ lines of code in rlang
• 12 line change to add to parser

S-expression parser

• very simple POC
• demonstrates use of non-native parsers

(+ 1 (* 2 3))

Interactive use

• Loosely based on Ross Ihaka’s R-script
• repl() function used to start REPL with new parser
  • repl(parse_text)
  • repl(tidy_parse_text)
  • repl(sexp_parse_text)
• q() used to exit

Package use

• Scripts - inst/*
• altparsers::src() to load scripts with parser
• export / document as normal

.onLoad <- function(...) {
  altparsers::src(system.file(package = "ex", "sexp"),
                  package = "ex",
                  altparsers::sexp_parse_file)
  
  altparsers::src(system.file(package = "ex", "tidy"),
                  package = "ex",
                  altparsers::tidy_parse_file)
}

Demo

altparsers package

• altparsers::src()
• altparsers::repl()
• Experiment with parsers in packages
• Parsers just simple examples
  • More complicated/useful features possible
  • Aviral Goel - Annotations for R