ES5 stand alone build

[arv]

We have an archaic build system that uses Nashorn (part of Java) to compile our JSX to to JS and then after that we run Closure Compiler on it. We initially started off using JSXTransformer and then migrated to babel for our jsx transformer. Babel is super slow (as you know). I'm trying to replace Babel with Sucrase as the jsx to js transformer and it was working fine up until 3.x. The code for this is at /~https://github.com/arv/sucrase-standalone.

Would you be interested in having an es5 stand alone build as part of the official sucrase package?

[alangpierce]

Hmm, I've certainly thought about an equivalent of babel-standalone, but I think I'd prefer it to target ES2018 so that Sucrase can build itself. So for now, I think it makes sense for it to be a separate project, and if it makes sense later (e.g. if it turns out to be broadly useful), I'd be happy to integrate it or mark it as "officially supported" or something like that.

What the trouble with Sucrase 3.x? Is the problem that Nashorn only runs ES5? I guess maybe I was wrong to assume that builds are run in relatively recent Node versions. Possibly a way to compile things cleanly down to ES5 is to clone the Sucrase repo and point Parcel at the src directory, using tsc as the compiler instead of Sucrase. Or you could find a way to transpile the code in node_modules, not sure if there are tricky details there.

I'm also a bit hesitant about the idea of standalone builds in the first place. I guess it's nice build system glue in various situations (like this one), but I've also seen people rely on them when it probably would be better to get the node_modules approach working.

Also worth noting: Sucrase could theoretically be a bit faster if it only focused on JSX. When I added support for imports and later flow and TS, I had to make the parser more complicated to handle scopes and the tricky ambiguities around < (either a less-than sign, a JSX tag start, or the start of an arrow function with a type parameter). A lot of that gets cleaner if you reduce scope to just JSX (possibly you could just tokenize instead of needing to parse as well). But still, Sucrase is obviously a lot faster than Babel already.

[arv]

The problem with 3.x is due to bugs in parcel. Parcel does not support object spread by default and there is a bug that prevents parcel from using non default transformers for code inside node_modules.

I'm currently converting to use tsc to generate es5 and then use webpack (but I guess I can use parcel on that too).

Regarding jsx only transformer. You still need a parser (since you need to know when regexps starts vs division) but like you said there is no need for the scope analysis.

It would be interesting to see if one could factor the code in a way that the logic of the parser and tokenizer could be shared without paying the price of ast and scopes when not needed.

I'm closing this for now since I think the issue is won't fix. Happy to keep the discussion going though.

[alangpierce]

Parcel does not support object spread by default

Got it, I also ran into some webpack trouble when I added the first use of object rest/spread. I considered just switching to Object.assign, bit it would be nice to "move forward, not backward", so to speak.

I bet you could send a PR to fix the parcel issue by just adding "objectRestSpread" to the babylon plugins here:
/~https://github.com/parcel-bundler/parcel/blob/master/src/assets/JSAsset.js#L68

Babylon has supported the syntax forever, it just needs to be flagged on.

Here are two bugs I found around it: parcel-bundler/parcel#1422 , parcel-bundler/parcel#1221

Given that it's officially in ES2018 and supported by node >= 8, I think any good JS parser these days should support it, even ones (like Acorn) that are hesitant to adopt future syntax.

Regarding jsx only transformer. You still need a parser (since you need to know when regexps starts vs division)

Babylon (and Acorn, from which it's forked) does some interesting stuff where it maintains lexer-specific state, and in particular an exprAllowed boolean that determines whether the current token is at the start of an expression. It is possible to do that without a full parse, at least in JS+JSX, I think. Then the tokenizer can say "/ at the start of an expression is a regex, otherwise division" and "< at the start of an expression is JSX, otherwise less-than". But at least in Babylon, later features (I think just Flow/TS, but maybe others) needed to have the parser reach down into the lexer to control the lexer state, so the lexer can't reliably be run in isolation. For Sucrase, I ended up getting rid of the lexer state and having the parser always guide the lexer, but if I really just cared about JSX, the lexer-only approach maybe could work and would probably be faster.

[alangpierce]

Closing, as mentioned, but happy to continue the conversation.