Lexer

open class Lexer : LexicalAnalyzer, CustomStringConvertible

Lexer provides a concrete implementation of the LexicalAnalyzer protocol and is used by default for the rest of the OysterKit stack (for example in Grammar.stream<N:Node,L:LexicalAnalyzer>(lexer:L)->AnySequence<N>).

In addition to the requirements of the LexicalAnalyzer protocol, this implementation provides the ability to skip characters from a CharacterSet when a call to mark() is made. This can be useful when the consumer always wants, for example, to ignore white space in a file.

  • If set, any characters in this set will be skipped when mark() is called

    Declaration

    Swift

    public var skip: CharacterSet?

  • Creates a new instance of Lexer for the supplied String.

    Declaration

    Swift

    public required init(source: String)

    Parameters

    source

    The String to perform the lexical analysis on

  • The integer offset of the current position in characters from the beginning of the String being scanned

    See also

    index

    Declaration

    Swift

    public var position: Int { get }

  • The index of the UnicodeScalarView position of the String being scanned

    Declaration

    Swift

    public var index: String.UnicodeScalarView.Index { get set }

  • The current depth of the Mark stack used for unwinding failed rules

    Declaration

    Swift

    public var depth: Int { get }

  • The character at the current scanning position

    Declaration

    Swift

    public var current: String { get }

  • Mark the position of the scanner. It should be noted that at this point any characters matching skip will be skipped. There should always be a matching call to either rewind() or proceed()->LexicalContext

    Declaration

    Swift

    public final func mark()

  • Mark the position of the scanner. It should be noted that at this point any characters matching skip will be skipped. There should always be a matching call to either rewind() or proceed()->LexicalContext

    Declaration

    Swift

    public func mark(skipping: Bool)

  • Unwind the current scanning state to the previous position. Typically used when a scan for a match has failed and it is necessary to try another match.

    See also

    proceed()->LexicalContext

    Declaration

    Swift

    open func rewind()

  • Creates a new LexicalContext from the start of top mark to the current scanner position. The mark is removed from the stack extending the range of the new top of the mark stack to this position.

    Declaration

    Swift

    open func proceed() -> LexicalContext

    Return Value

    A LexicalContext for the current match

  • Removes the top most Mark from the stack without creating a new LexicalContext effectively advancing the scanning position of the new topmost mark to this position.

    Declaration

    Swift

    open func consume()

  • Removes the top most Mark from the stack without creating a new LexicalContext effectively advancing the scanning position of the new topmost mark to this position.

    Declaration

    Swift

    open func fastForward() -> LexicalContext

  • true if the end of the String being scanned has been reached

    Declaration

    Swift

    public var endOfInput: Bool { get }

  • Scans until the specified terminal is reached in the source String. If the end of the String is reached before the supplied String is matched a GrammarError is thrown. If the terminal is matched, the scanning position will be at the end of that match.

    See also

    scanUpTo(terminal:String)

    Declaration

    Swift

    open func scan(terminal: String) throws

    Parameters

    terminal

    A string to scan for

  • Scans until one of the characters in the supplied CharacterSet is found. If the end of the source is reached before a character from the set is found a GrammarError is thrown. The scanner position will be directly after the matched character.

    Declaration

    Swift

    open func scan(oneOf: CharacterSet) throws

    Parameters

    oneOf

    A CharacterSet to scan for

  • Scans up to (that is, the position of the scanner will be at the start of the match, not after it) the supplied String. If the terminal is not found a GrammarError is thrown. The scanner position will be at the first character of the matched terminal.

    See also

    scan(terminal:String)

    Declaration

    Swift

    open func scanUpTo(terminal: String) throws

    Parameters

    terminal

    A string to scan for

  • Scans up to one of the characters in the supplied CharacterSet is found. If the end of the source is reached before a character from the set is found a GrammarError is thrown. The scanner position will be directly at the matched character.

    Declaration

    Swift

    open func scanUpTo(oneOf terminal: CharacterSet) throws

    Parameters

    oneOf

    A CharacterSet to scan for

  • Scan the supplied regular expression (NSRegularExpression). If not found an Error should be the thrown. The scanner position should be directly after the matched pattern.

    Declaration

    Swift

    open func scan(regularExpression regex: NSRegularExpression) throws

    Parameters

    regularExpression

    The NSRegularExpression to scan for

  • Advances the scanner one character. If the end of the source String has already been reached a GrammarException will be thrown.

    Declaration

    Swift

    open func scanNext() throws

  • Returns the String from the source representing the supplied range.

    Declaration

    Swift

    public subscript(range: Range<String.UnicodeScalarView.Index>) -> String { get }

  • The String currently being scanned

    Declaration

    Swift

    public var source: String { get }

  • A description of the current state of the Lexer

    Declaration

    Swift

    public var description: String { get }