Add support for first-class expressions to Terraform lexer

[pyrmont]

Terraform 0.12 adds support for first-class expressions. Prior versions of Terraform required all non-literal expressions to be included as interpolated sequences inside strings.

In addition, the demo and visual samples have been rationalised. This will assist in future updates to the Terraform lexer.

This fixes #1301.

[pyrmont]

@sashee My solution was to add rules for certain punctuation characters to the :hash and :array states that the Terraform lexer inherits from the HCL lexer:

rouge/lib/rouge/lexers/hcl.rb

Lines 113 to 129 in 30e1e8c

	state :hash do
	mixin :comments_and_whitespace
	rule %r/\=/, Punctuation
	rule %r/\}/, Punctuation, :pop!
	mixin :root
	end
	state :array do
	mixin :comments_and_whitespace
	rule %r/,/, Punctuation
	rule %r/\]/, Punctuation, :pop!
	mixin :root
	end

Another solution would be to add the rule to the :root state. Would this be preferable? Can expressions occur outside objects? I was afraid that adding to :root might have unintended consequences but perhaps there are situations outside of :hash and :array where first-class expressions should be permitted.

What do you think?

[sashee]

@pyrmont, I've just checked some of my terraform modules and I see another problem (I think it's related):

resource "aws_acm_certificate_validation" "cert" {
  certificate_arn         = "${aws_acm_certificate.cert.arn}"
  validation_record_fqdns = aws_route53_record.cert_validation[*].fqdn
}

The [*] part is reported as error, but that is valid.

If I wrap it in quotes: validation_record_fqdns = "${aws_route53_record.cert_validation[*].fqdn}" then it is not reported.

Apart from this issue all the other places are working fine now. I don't think expressions can occur outside objects.

[pyrmont]

@sashee Yeah, that's because I didn't add * to the character class that can match in a first-class expression. What characters should it include? I have [.,()]. Is * all I need to add?

[sashee]

Yes, I think the only special character is *.