From cdede47db2ff0de99544a1991e096f0e8c75d644 Mon Sep 17 00:00:00 2001
From: Julian Simioni <julian@simioni.org>
Date: Mon, 15 Mar 2021 10:21:34 -0700
Subject: [PATCH] feat(output): Rename 'Fail' to 'Expected Failures' in test
 output

Background
==========

The fuzzy tester has a bit of added functionality not present in many
testing tools: a test can be expected to pass, or expected to fail.

This is because a geocoder is a complex machine and we can't always make
a certain test pass.

We often use this when creating
tests for new functionality we haven't yet implemented. Marking the
tests as passing then becomes a bit of a celebration of a job completed
:)

On the other hand it might be because there's a known regression that
we can't currently fix, and have deemed acceptable. Maybe it came along
with an improvement in another area that is considered "worth it", or we
just plain don't know how to fix it yet.

This leads to 4 possible outcomes for any individual test. We currently
refer to them as follows:

**Passed:** the test was expected to pass and it did pass
**Improvement:** the test was not expected to pass, but it passed (usually
as a result of a bug fix or new feature)
**Fail**: The test was expected to fail, and it failed
**Regression**: The test was expected to pass, but it failed. This is
usually a problem and we look at these closely

The Problem
===========

Even for those of us familiar with the fuzzy tester, remembering if it's
'regressions' or 'fails' that represent a _new_, unexpected failure, can
sometimes be a challenge.

When viewing test output directly, the difference in color can be
helpful, but depending on how the tests are run, colors in the output
might not be available.

What's changed
==============

This PR replaces 'Fails' with 'Expected Failures' so that it's very
clear which of the two types of failures are which.
---
 output_generators/autocomplete.js | 2 +-
 output_generators/terminal.js     | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/output_generators/autocomplete.js b/output_generators/autocomplete.js
index a61745c..a2b58f5 100644
--- a/output_generators/autocomplete.js
+++ b/output_generators/autocomplete.js
@@ -92,7 +92,7 @@ function prettyPrintSuiteResults( suiteResults, config, testSuites ){
   console.log( '\nAggregate test results'.blue );
   console.log( 'Pass: ' + suiteResults.stats.pass.toString().green );
   console.log( 'Improvements: ' + suiteResults.stats.improvement.toString().green );
-  console.log( 'Fail: ' + suiteResults.stats.fail.toString().yellow );
+  console.log( 'Expected Failures: ' + suiteResults.stats.fail.toString().yellow );
   console.log( 'Placeholders: ' + suiteResults.stats.placeholder.toString().cyan );
 
   var numRegressions = suiteResults.stats.regression;
diff --git a/output_generators/terminal.js b/output_generators/terminal.js
index 0b789c0..990cf55 100644
--- a/output_generators/terminal.js
+++ b/output_generators/terminal.js
@@ -124,7 +124,7 @@ function prettyPrintSuiteResults( suiteResults, config, testSuites ){
   console.log( '\nAggregate test results'.blue );
   console.log( 'Pass: ' + suiteResults.stats.pass.toString().green );
   console.log( 'Improvements: ' + suiteResults.stats.improvement.toString().green);
-  console.log( 'Fail: ' + suiteResults.stats.fail.toString().yellow );
+  console.log( 'Expected Failures: ' + suiteResults.stats.fail.toString().yellow );
   console.log( 'Placeholders: ' + suiteResults.stats.placeholder.toString().cyan );
 
   var numRegressions = suiteResults.stats.regression;