Skip to content

Commit

Permalink
Fix tokenization of number literals in YAML lexer (#1239)
Browse files Browse the repository at this point in the history
Stand-alone number literals in a block scalar are being tokenized as
`Literal::String` not as `Literal::Number`.

This commit fixes that while also ensuring that, if those literals are
preceded or succeeded by non-whitespace characters (that aren't
comments), they are tokenized as `Literal::String`.

Finally, the commit includes a change to ensure times are tokenized as
`Literal::String` (which is the existing behavior).
  • Loading branch information
ashmaroli authored and pyrmont committed Jul 1, 2019
1 parent c5b7798 commit 28f484e
Show file tree
Hide file tree
Showing 2 changed files with 4 additions and 1 deletion.
4 changes: 3 additions & 1 deletion lib/rouge/lexers/yaml.rb
Original file line number Diff line number Diff line change
Expand Up @@ -328,7 +328,7 @@ def set_indent(match, opts={})
state :plain_scalar_in_block_context do
# the : indicator ends a scalar
rule %r/[ ]*(?=:[ \n]|:$)/, Text, :pop!
rule %r/[ ]*:/, Str
rule %r/[ ]*:\S+/, Str
rule %r/[ ]+(?=#)/, Text, :pop!
rule %r/[ ]+$/, Text
# check for new documents or dedents at the new line
Expand All @@ -339,6 +339,8 @@ def set_indent(match, opts={})

rule %r/[ ]+/, Str
rule SPECIAL_VALUES, Name::Constant
rule %r/\d+(?:\.\d+)?(?=(\r?\n)| +#)/, Literal::Number, :pop!

# regular non-whitespace characters
rule %r/[^\s:]+/, Str
end
Expand Down
1 change: 1 addition & 0 deletions spec/visual/samples/yaml
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ national:
name: Mark McGwire
hr: 65
avg: 0.278
addr: B 221 Block
-
name: Sammy Sosa
hr: 63
Expand Down

0 comments on commit 28f484e

Please sign in to comment.