Support for ansi colour coding in the test window

[bruderstein]

Some assertion libraries (for example unexpected) can use ansi terminal colour coding in the test results to highlight a diff between expected and actual.

When running the tests in webstorm terminal via the mocha command line, we can see these beautiful diffs, in the wallaby "failing tests" window however, we can't get the colours (when forced to output ansi, they appear as just �[31m�[1m... etc.

It would be awesome to support these colour codes - it's actually a reason i still occasionally run the tests via the command line to see the output.

Or from the webstorm console:

And that is shown in wallaby as:

[ArtemGovorov]

Slightly off topic, but wallaby.js supports showing nice diffs for assertion libraries that throw errors with expected, actual and showDiff properties. I'm not sure where the convention originates from, but expect.js, chai, etc., support it. This way mocha, wallaby, and potentially other test runners may do the diff and display it nicely.

For example, for chai, expect.js, and other assertion libs that follow the convention, wallaby can do this:

I'm wondering if unexpected.js would be open to the idea of following the convention and adding these properties to the assertion errors they throw.

[bruderstein]

While this works for things that can be compared in text and line by line, this doesn't work for many of the things that unexpected supports, such as diffing custom types. The diffs are created internally, which means you can assert specifics about two objects, without requiring them to be otherwise identical.

An excellent example is the unexpected-dom library, which allows assertions over the DOM.

Here I'm asserting that a particular CSS class is included in the node:

(Also note the syntax highlighting of the HTML, which you get for free because the diffing algorithm understands the DOM, and can highlight it as it goes 😄)
Here there is no "expected" result to give out, we are expecting that the class is on the node, we don't define the whole node. This is conducive to writing good, non-brittle tests, as we only need to assert over the things that are actually important for the test in question. Adding the actual class as the "actual" value doesn't make sense either, as that may be much longer than what we are expecting.

Actual   
row col-4 Select__arrow Select__arrow--down Select__box--open 

Expected
Select__box--closed

It also allows assertions like 'to satisfy', meaning that the object contains at least the given properties.
e.g.

var actualObject = {
   some: 'prop'
   important: 42,
   alsoImportant: [
        { id: 1, name: 'Herr von Neumann'}, 
        { id: 22, name: 'Herr A. Einstein', profession: 'genius' }
   ]
};

// check the object contains at least what we need
expect(actualObject, 'to satisfy', { important: 42, alsoImportant: [ { id: 1}, { id: 22} ]);

It's not possible to produce a line-by-line diff for this externally, because it can't be known what the rules for matching are, without reimplementing it.

I'm pretty sure that's why unexpected creates its diffs itself, such that complex types can be diffed.

[ArtemGovorov]

Fair enough Dave, thanks for the explanation. Will have a look what can be done.

[papandreou]

Unexpected also supports generating HTML error messages (including diffs) for the browser, and this representation can be enriched with media that cannot be shown in a terminal, such as http://unexpected.js.org/unexpected-resemble/

I wanted to try playing with/cheering for getting those HTML errors into Atom once wallaby adds support for it :)

[ArtemGovorov]

@papandreou Nice! I just thought how to display the diffs in Atom.

Would be totally awesome if unexpected was also supporting returning the diff in a form a string plus a token array (describing what parts of the string are what). Something like language grammar parsers do, so that the content and the presentation would be separated and one could create any kind of presentation for the content.

[papandreou]

@ArtemGovorov Internally Unexpected uses /~https://github.com/sunesimonsen/magicpen to build its output, and that includes something like the token representation that you're asking for. It's exposed via the err.getErrorMessage(<format>) method. However, it's a bit of a low-level representation, and since the error messages and diffs themselves can vary greatly depending on the output format, you have to specify the desired target format (text, ansi, html, or even coloredConsole) up front when building the error message, so I would suggest that you rely on that instead:

var expect = require('unexpected');
try {
    expect('abc', 'to equal', 'abe');
} catch (err) {
    err.getErrorMessage('html'); // returns magicpen instance with the tokens
    err.getErrorMessage('html').toString(); // returns HTML serialization
    err.getErrorMessage('ansi').toString(); // returns ansi serialization
    err.getErrorMessage('text').toString(); // returns text serialization
}

... Or does wallaby actually need to understand the contents of the error message/diff? If so, please write a few words about why, then we'll see what we can do.

[sunesimonsen]

@ArtemGovorov I guess we can make a serializer for magicpen that would produce should an output. But for this use case I think it would be preferable to show the error.htmlMessage as rendered html if it is available - that would probably be pretty easy to support from wallabyjs.

[ArtemGovorov]

From what I can see, err.getErrorMessage(<format>) may be enough. It does return tokens with styles as you've said, so that I can just build own output.

Wallaby does need to understand the contents style semantics, to be able to format the output with a custom formatter. For example, for the IntelliJ/Visual Studio platform I'll be reading the tokens array and rendering them into Failing Tests console (which is a micro instance of the IDE editor class), using current theme colours. Same for Atom, as opposed to the built-in html serialisation (which hardcodes styles/colors), we'll need to use CSS classes (that will support Atom theme changes nicely).

[papandreou]

@ArtemGovorov What are the capabilities of that terminal? Maybe you could write an IntelliJ/Visual Studio serializer for magicpen? I think that would be easier to maintain than relying on the internal representation of the tokens. "Serializer" is actually a bit of a misnomer, as the toString method can take arbitrary arguments and doesn't really need to return a string.

Then you could do something like this:

err.getErrorMessage('intelliJ').toString(theIntelliJConsole);

[ArtemGovorov]

@papandreou For IntelliJ it's basically a printer, like:

public interface Printer {
  void print(String text, ConsoleViewContentType contentType);
}

(where ConsoleViewContentType encapsulates styles).

So it has to be used as a sequence of the printer instance calls, passing parts of the message and styles (like colours) for the parts. Those styles are theme-able, so that if exposed, the IDE user can configure them and theme authors may consider them as well.

Ideally, it'd be good to map styles to the IDE editor styles for the majority of token types (tags, attributes, JS strings, errors and other tokens) so that the output looks and feels like in the IDE editor and easily customisable by the end user.

[sunesimonsen]

@ArtemGovorov I fully agree with @papandreou adding a serializer to magicpen is the right way to go about it. It would be possible to generate a string with additional token information that could be styled the way you like it. It would also be possible to produce html with class names that can be styled from CSS.

[sunesimonsen]

@ArtemGovorov as you can see here: /~https://github.com/unexpectedjs/unexpected/blob/master/lib/styles.js unexpected uses logical names for most things as theme them differently if it is html or ansi output. But you could produce output with the logical names instead or map them to names that IntelliJ understands.

[ArtemGovorov]

@sunesimonsen Cool, thanks. Could you please point me to the right direction in terms of how to write the magicpen serializer and install it to unexpected? I'll probably just pass the logical names coming from unexpected and its plugins to an IDE/editor, so that the logic of producing the output with actual styles resides in the target IDE. This way IntelliJ plugin will use its printer, Atom package will produce HTML with proper CSS classes, etc.

[sunesimonsen]

@ArtemGovorov I'm afraid that there is no extensive documentation on the subject, but try to take a look at this code: /~https://github.com/sunesimonsen/magicpen/blob/master/lib/ColoredConsoleSerializer.js and this is where they get installed: /~https://github.com/sunesimonsen/magicpen/blob/master/lib/MagicPen.js#L61-L69 you can just add one more to the hash.

I guess it is must be something like:

• flatten the blocks using flattenBlocksInLines(lines)
• run though each of the lines and their output entries and produce a plain text output and an array of styling tokens.

I it cool if you would give it a go, otherwise I can see if I can get some time to help you out.

[ArtemGovorov]

@sunesimonsen Thanks! I'll have a look.

For now as a quick solution for the IntelliJ platform, I'll just add an ansi decoder to display whatever colors are coming from unexpected. But will definitely need to implement the serializer for the theme-friendly solution for all supported IDEs.

[sunesimonsen]

@ArtemGovorov sounds awesome - thanks a lot!

[ArtemGovorov]

As originally requested in the issue, for now implemented unexpected.js support via a simple IntelliJ built-in ANSI decoder: v1.0.27 of the plugin and v1.0.87 of the core. Looks nice in both default and Darcula themes:

Will implement the discussed magicpen serializer later.

[sunesimonsen]

That is awesome, thanks!

[bruderstein]

That is amazingly awesome. Thank you so much, everyone!

[bruderstein]

This looks great now. The only thing is the inline message (inline with the code), is still with the ansi escape sequences. Not sure how easy that is to fix. The important thing works great.

[ArtemGovorov]

@bruderstein Are you still forcing unexpected to output ansi? If so, don't force it and it should work fine (like you can see on my screenshots). If it doesn't work, please share a sample where I could reproduce the issue.

[bruderstein]

Sorry, yes, I was, due to dom emulation. I'll solve that a different way. Thank you.

And sorry for the noise.

[hellboy81]

Is there colored output support in atom plugin?

[ArtemGovorov]

@hellboy81 Not in Atom yet.