Skipped tests are classified as passed

[williamdenton]

there is no special handling for Outcome.Skipped so skipped tests appear as successful in xml output.

They need a <skipped/> element so tools like jenkins can recognise and classify the tests appropriately

junit.testlogger/src/JUnit.Xml.TestLogger/JUnitXmlTestLogger.cs

Lines 436 to 456 in 23bbc8f

	if (result.Outcome == TestOutcome.Failed)
	{
	var failureBodySB = new StringBuilder();
	if (this.FailureBodyFormatOption == FailureBodyFormat.Verbose)
	{
	failureBodySB.AppendLine(result.ErrorMessage);
	// Stack trace included to mimic the normal test output
	failureBodySB.AppendLine("Stack Trace:");
	}
	failureBodySB.AppendLine(result.ErrorStackTrace);
	var failureElement = new XElement("failure", failureBodySB.ToString());
	failureElement.SetAttributeValue("type", "failure"); // TODO are there failure types?
	failureElement.SetAttributeValue("message", result.ErrorMessage);
	testcaseElement.Add(failureElement);
	}

[williamdenton]

closest i could find to a spec for junit output for a test case:

/~https://github.com/behave/behave/blob/85bca4533ae4fb0cb27b65d04c5dd50355210395/etc/junit.xml/junit-4.xsd#L38-L53

    <xs:element name="testcase">
        <xs:complexType>
            <xs:sequence>
                <xs:element ref="skipped" minOccurs="0" maxOccurs="1"/>
                <xs:element ref="error" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="failure" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="system-out" minOccurs="0" maxOccurs="unbounded"/>
                <xs:element ref="system-err" minOccurs="0" maxOccurs="unbounded"/>
            </xs:sequence>
            <xs:attribute name="name" type="xs:string" use="required"/>
            <xs:attribute name="assertions" type="xs:string" use="optional"/>
            <xs:attribute name="time" type="xs:string" use="optional"/>
            <xs:attribute name="classname" type="xs:string" use="optional"/>
            <xs:attribute name="status" type="xs:string" use="optional"/>
        </xs:complexType>
    </xs:element>

[williamdenton]

for now im using /~https://github.com/spekt/xunit.testlogger with vstest then using /~https://github.com/gabrielweyer/xunit-to-junit to convert xunit to junit. This gives ideal output in jenkins, with nice test names and skipped results

[Siphonophora]

@williamdenton Thanks. I will take a look. I couldn't find a spec that included skipped, which is why I grouped skipped under passed.

[Siphonophora]

@williamdenton Apologies for the delay. The new pre-release package includes a resolution for this issue. If you have time to take a look and provide feedback, that would be great.

[williamdenton]

Hi @Siphonophora , ive just come round to modifying my CI pipeline again and pulled in your package from myget. Works well! Thank you!

Is this in master/nuget yet? (it took me a long time to try it out)

Greatly appreciate you fixing this 👍

[Siphonophora]

@williamdenton I'm glad its work. Its not on nuget yet. @codito Can we go ahead and push the current version to nuget?

[codito]

@Siphonophora @williamdenton Done. Latest package is here: https://www.nuget.org/packages/JunitXml.TestLogger/2.1.32

[williamdenton]

Thanks, once again i really appreciate you maintaining this for the community! (do you do release notes?)

[Siphonophora]

@williamdenton Yes, I will write up release notes here later today or tomorrow: /~https://github.com/spekt/junit.testlogger/releases

Edit: here is the update:
Resolved: /~https://github.com/spekt/junit.testlogger/releases/tag/v2.1.32