Fixes microsoft/monaco-editor#1818: check matchOnlyAtLineStart when e…

[bolinfest]

This PR fixes microsoft/monaco-editor#1818.

I tested this by copying one of the .html files in playground.generated and changing the JavaScript to include the following:

const DEMO_LANG_ID = "demo";

const languageConfiguration = {
  // The main tokenizer for our languages
  tokenizer: {
    root: [
      [/^\!/, { token: 'delimiter.curly', next: 'jsonInBang', nextEmbedded: 'json' }],
    ],
    jsonInBang: [
      [/^\!/, { token: 'delimiter.curly', next: '@pop', nextEmbedded: '@pop' }],
    ],
  },
};
monaco.languages.register({
  id: DEMO_LANG_ID,
  extensions: ['.example'],
});
monaco.languages.setMonarchTokensProvider(DEMO_LANG_ID, languageConfiguration);

      const value = `\
!
  {
    "foo": "b!ar"
  }
!
`;

var editor = monaco.editor.create(document.getElementById("container"), {
  value,
  language: DEMO_LANG_ID,

  lineNumbers: "off",
  roundedSelection: false,
  scrollBeyondLastLine: false,
  readOnly: false,
  theme: "vs-dark",
});

When I loaded the page, now the entire "b!ar" string literal was highlighted correctly. Previously, the syntax highlighting stopped at the !.

[bolinfest]

/~https://github.com/microsoft/monaco-editor/blob/master/CONTRIBUTING.md mentions a bunch of test pages, but not a lot about running automated tests. I did try running the following from my ~/src/vscode folder as it recommends:

npm run monaco-editor-test

but that failed with:

npm ERR! missing script: monaco-editor-test

npm ERR! A complete log of this run can be found in:
npm ERR!     /Users/mbolin/.npm/_logs/2020-02-07T23_54_12_536Z-debug.log

[alexdima]

Thank you! The change looks good.

I ran the tests at /~https://github.com/microsoft/monaco-languages using the patch and they all pass.

[bolinfest]

@alexdima Thanks so much for the fast accept and the release!

If I want to add a test, is monaco-languages the right place to do it?

[alexdima]

The best would be to have tests inside the vscode repository... The monaco-languages repository is more concerned with testing that the various tokenizers are correct in tokenizing their respective programming languages...

But currently there are no specific Monarch tests in the vscode repository so you would need to scaffold a bit to get some tests running...