Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 16 additions & 0 deletions doc/api/cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -2701,6 +2701,21 @@ added: v22.8.0
Require a minimum percent of covered lines. If code coverage does not reach
the threshold specified, the process will exit with code `1`.

### `--test-coverage-statements=threshold`

<!-- YAML
added: REPLACEME
-->

> Stability: 1 - Experimental

Require a minimum percent of covered statements. If code coverage does not reach
the threshold specified, the process will exit with code `1`.

Statement coverage uses acorn to parse source files and extract statement
nodes from the AST. The V8 coverage ranges are then mapped to these statements
to determine which ones were executed.

### `--test-force-exit`

<!-- YAML
Expand Down Expand Up @@ -3687,6 +3702,7 @@ one is included in the list below.
* `--test-coverage-functions`
* `--test-coverage-include`
* `--test-coverage-lines`
* `--test-coverage-statements`
* `--test-global-setup`
* `--test-isolation`
* `--test-name-pattern`
Expand Down
11 changes: 11 additions & 0 deletions doc/api/test.md
Original file line number Diff line number Diff line change
Expand Up @@ -3255,12 +3255,15 @@ are defined, while others are emitted in the order that the tests execute.
* `totalLineCount` {number} The total number of lines.
* `totalBranchCount` {number} The total number of branches.
* `totalFunctionCount` {number} The total number of functions.
* `totalStatementCount` {number} The total number of statements.
* `coveredLineCount` {number} The number of covered lines.
* `coveredBranchCount` {number} The number of covered branches.
* `coveredFunctionCount` {number} The number of covered functions.
* `coveredStatementCount` {number} The number of covered statements.
* `coveredLinePercent` {number} The percentage of lines covered.
* `coveredBranchPercent` {number} The percentage of branches covered.
* `coveredFunctionPercent` {number} The percentage of functions covered.
* `coveredStatementPercent` {number} The percentage of statements covered.
* `functions` {Array} An array of functions representing function
coverage.
* `name` {string} The name of the function.
Expand All @@ -3273,22 +3276,30 @@ are defined, while others are emitted in the order that the tests execute.
numbers and the number of times they were covered.
* `line` {number} The line number.
* `count` {number} The number of times the line was covered.
* `statements` {Array} An array of statements representing statement
coverage.
* `line` {number} The line number where the statement starts.
* `count` {number} The number of times the statement was executed.
* `thresholds` {Object} An object containing whether or not the coverage for
each coverage type.
* `function` {number} The function coverage threshold.
* `branch` {number} The branch coverage threshold.
* `line` {number} The line coverage threshold.
* `statement` {number} The statement coverage threshold.
* `totals` {Object} An object containing a summary of coverage for all
files.
* `totalLineCount` {number} The total number of lines.
* `totalBranchCount` {number} The total number of branches.
* `totalFunctionCount` {number} The total number of functions.
* `totalStatementCount` {number} The total number of statements.
* `coveredLineCount` {number} The number of covered lines.
* `coveredBranchCount` {number} The number of covered branches.
* `coveredFunctionCount` {number} The number of covered functions.
* `coveredStatementCount` {number} The number of covered statements.
* `coveredLinePercent` {number} The percentage of lines covered.
* `coveredBranchPercent` {number} The percentage of branches covered.
* `coveredFunctionPercent` {number} The percentage of functions covered.
* `coveredStatementPercent` {number} The percentage of statements covered.
* `workingDirectory` {string} The working directory when code coverage
began. This is useful for displaying relative path names in case the tests
changed the working directory of the Node.js process.
Expand Down
168 changes: 167 additions & 1 deletion lib/internal/test_runner/coverage.js
Original file line number Diff line number Diff line change
Expand Up @@ -30,6 +30,10 @@ const { tmpdir } = require('os');
const { join, resolve, relative } = require('path');
const { fileURLToPath, URL } = require('internal/url');
const { kMappings, SourceMap } = require('internal/source_map/source_map');
const { Parser: AcornParser } =
require('internal/deps/acorn/acorn/dist/acorn');
const { simple: acornWalkSimple } =
require('internal/deps/acorn/acorn-walk/dist/walk');
const {
codes: {
ERR_SOURCE_MAP_CORRUPT,
Expand All @@ -39,6 +43,12 @@ const {
const { matchGlobPattern } = require('internal/fs/glob');
const { constants: { kMockSearchParam } } = require('internal/test_runner/mock/loader');

// Statement types excluded from coverage: containers (BlockStatement)
// and empty statements that carry no executable semantics.
const kExcludedStatementTypes = new SafeSet([
'BlockStatement', 'EmptyStatement',
]);

const kCoverageFileRegex = /^coverage-(\d+)-(\d{13})-(\d+)\.json$/;
const kIgnoreRegex = /\/\* node:coverage ignore next (?<count>\d+ )?\*\//;
const kLineEndingRegex = /\r?\n$/u;
Expand Down Expand Up @@ -69,6 +79,64 @@ class TestCoverage {
}

#sourceLines = new SafeMap();
#sourceStatements = new SafeMap();

getStatements(fileUrl, source) {
if (this.#sourceStatements.has(fileUrl)) {
return this.#sourceStatements.get(fileUrl);
}

try {
source ??= readFileSync(fileURLToPath(fileUrl), 'utf8');
} catch {
// The file can no longer be read. Leave it out of statement coverage.
this.#sourceStatements.set(fileUrl, null);
return null;
}

const statements = [];

// acorn-walk's simple() fires a generic "Statement" visitor for every
// node dispatched in a statement position (Program body, block bodies,
// if/for/while bodies, etc.). This automatically covers all current and
// future ESTree statement types without hardcoding a list.
const visitor = { __proto__: null };
visitor.Statement = (node) => {
if (kExcludedStatementTypes.has(node.type)) return;
ArrayPrototypePush(statements, {
__proto__: null,
startOffset: node.start,
endOffset: node.end,
count: 0,
});
};
Comment on lines +99 to +112
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You should create this object inline


// Parse as script with permissive flags — script mode is non-strict,
// so it handles legacy CJS (e.g. `with` statements) while the
// allow* flags enable ESM syntax (import/export/top-level await).
let ast;
try {
ast = AcornParser.parse(source, {
__proto__: null,
ecmaVersion: 'latest',
sourceType: 'script',
allowHashBang: true,
allowReturnOutsideFunction: true,
allowImportExportEverywhere: true,
allowAwaitOutsideFunction: true,
});
} catch {
// Acorn could not parse the file (e.g. non-JS syntax, TypeScript).
// Degrade gracefully — the file will report no statement coverage.
this.#sourceStatements.set(fileUrl, null);
return null;
}

acornWalkSimple(ast, visitor);

this.#sourceStatements.set(fileUrl, statements);
return statements;
}

getLines(fileUrl, source) {
// Split the file source into lines. Make sure the lines maintain their
Expand Down Expand Up @@ -145,18 +213,22 @@ class TestCoverage {
totalLineCount: 0,
totalBranchCount: 0,
totalFunctionCount: 0,
totalStatementCount: 0,
coveredLineCount: 0,
coveredBranchCount: 0,
coveredFunctionCount: 0,
coveredStatementCount: 0,
coveredLinePercent: 0,
coveredBranchPercent: 0,
coveredFunctionPercent: 0,
coveredStatementPercent: 0,
},
thresholds: {
__proto__: null,
line: this.options.lineCoverage,
branch: this.options.branchCoverage,
function: this.options.functionCoverage,
statement: this.options.statementCoverage,
},
};

Expand All @@ -174,7 +246,17 @@ class TestCoverage {
const functionReports = [];
const branchReports = [];

const lines = this.getLines(url);
// Read source once and pass to both getLines and getStatements to
// avoid double disk I/O for the same file.
let source;
try {
source = readFileSync(fileURLToPath(url), 'utf8');
} catch {
// The file can no longer be read. Skip it entirely.
continue;
}

const lines = this.getLines(url, source);
if (!lines) {
continue;
}
Expand Down Expand Up @@ -243,29 +325,88 @@ class TestCoverage {
}
}

// Compute statement coverage by mapping V8 ranges to AST statements.
// Pass the source already read above to avoid double disk I/O.
const statements = this.getStatements(url, source);
let totalStatements = 0;
let statementsCovered = 0;
const statementReports = [];

if (statements) {
// Flatten all V8 coverage ranges into a single array so
// the statement loop only needs two levels of iteration.
const allRanges = [];
for (let fi = 0; fi < functions.length; ++fi) {
const { ranges } = functions[fi];
for (let ri = 0; ri < ranges.length; ++ri) {
ArrayPrototypePush(allRanges, ranges[ri]);
}
}

for (let j = 0; j < statements.length; ++j) {
const stmt = statements[j];
let bestCount = 0;
let bestSize = Infinity;

for (let ri = 0; ri < allRanges.length; ++ri) {
const range = allRanges[ri];
if (doesRangeContainOtherRange(range, stmt)) {
const size = range.endOffset - range.startOffset;
if (size < bestSize) {
bestCount = range.count;
bestSize = size;
}
}
}

stmt.count = bestSize !== Infinity ? bestCount : 0;

const stmtLine = findLineForOffset(stmt.startOffset, lines);
const isIgnored = stmtLine != null && stmtLine.ignore;

if (!isIgnored) {
totalStatements++;
ArrayPrototypePush(statementReports, {
__proto__: null,
line: stmtLine?.line,
count: stmt.count,
});
if (stmt.count > 0) {
statementsCovered++;
}
}
}
}

ArrayPrototypePush(coverageSummary.files, {
__proto__: null,
path: fileURLToPath(url),
totalLineCount: lines.length,
totalBranchCount: totalBranches,
totalFunctionCount: totalFunctions,
totalStatementCount: totalStatements,
coveredLineCount: coveredCnt,
coveredBranchCount: branchesCovered,
coveredFunctionCount: functionsCovered,
coveredStatementCount: statementsCovered,
coveredLinePercent: toPercentage(coveredCnt, lines.length),
coveredBranchPercent: toPercentage(branchesCovered, totalBranches),
coveredFunctionPercent: toPercentage(functionsCovered, totalFunctions),
coveredStatementPercent: toPercentage(statementsCovered, totalStatements),
functions: functionReports,
branches: branchReports,
lines: lineReports,
statements: statementReports,
});

coverageSummary.totals.totalLineCount += lines.length;
coverageSummary.totals.totalBranchCount += totalBranches;
coverageSummary.totals.totalFunctionCount += totalFunctions;
coverageSummary.totals.totalStatementCount += totalStatements;
coverageSummary.totals.coveredLineCount += coveredCnt;
coverageSummary.totals.coveredBranchCount += branchesCovered;
coverageSummary.totals.coveredFunctionCount += functionsCovered;
coverageSummary.totals.coveredStatementCount += statementsCovered;
}

coverageSummary.totals.coveredLinePercent = toPercentage(
Expand All @@ -280,6 +421,10 @@ class TestCoverage {
coverageSummary.totals.coveredFunctionCount,
coverageSummary.totals.totalFunctionCount,
);
coverageSummary.totals.coveredStatementPercent = toPercentage(
coverageSummary.totals.coveredStatementCount,
coverageSummary.totals.totalStatementCount,
);
coverageSummary.files.sort(sortCoverageFiles);

return coverageSummary;
Expand Down Expand Up @@ -695,4 +840,25 @@ function doesRangeContainOtherRange(range, otherRange) {
range.endOffset >= otherRange.endOffset;
}

function findLineForOffset(offset, lines) {
let start = 0;
let end = lines.length - 1;

while (start <= end) {
const mid = MathFloor((start + end) / 2);
const line = lines[mid];

if (offset >= line.startOffset && offset <= line.endOffset) {
return line;
}
if (offset > line.endOffset) {
start = mid + 1;
} else {
end = mid - 1;
}
}

return null;
}

module.exports = { setupCoverage, TestCoverage };
3 changes: 3 additions & 0 deletions lib/internal/test_runner/test.js
Original file line number Diff line number Diff line change
Expand Up @@ -1367,6 +1367,9 @@ class Test extends AsyncResource {

{ __proto__: null, actual: coverage.totals.coveredFunctionPercent,
threshold: this.config.functionCoverage, name: 'function' },

{ __proto__: null, actual: coverage.totals.coveredStatementPercent,
threshold: this.config.statementCoverage, name: 'statement' },
];

for (let i = 0; i < coverages.length; i++) {
Expand Down
11 changes: 9 additions & 2 deletions lib/internal/test_runner/utils.js
Original file line number Diff line number Diff line change
Expand Up @@ -227,6 +227,7 @@ function parseCommandLine() {
let lineCoverage;
let branchCoverage;
let functionCoverage;
let statementCoverage;
let destinations;
let isolation;
let only = getOptionValue('--test-only');
Expand Down Expand Up @@ -318,10 +319,12 @@ function parseCommandLine() {
branchCoverage = getOptionValue('--test-coverage-branches');
lineCoverage = getOptionValue('--test-coverage-lines');
functionCoverage = getOptionValue('--test-coverage-functions');
statementCoverage = getOptionValue('--test-coverage-statements');

validateInteger(branchCoverage, '--test-coverage-branches', 0, 100);
validateInteger(lineCoverage, '--test-coverage-lines', 0, 100);
validateInteger(functionCoverage, '--test-coverage-functions', 0, 100);
validateInteger(statementCoverage, '--test-coverage-statements', 0, 100);
}

if (rerunFailuresFilePath) {
Expand Down Expand Up @@ -351,6 +354,7 @@ function parseCommandLine() {
branchCoverage,
functionCoverage,
lineCoverage,
statementCoverage,
only,
reporters,
setup,
Expand Down Expand Up @@ -449,8 +453,11 @@ function formatUncoveredLines(lines, table) {
return ArrayPrototypeJoin(lines, ', ');
}

const kColumns = ['line %', 'branch %', 'funcs %'];
const kColumnsKeys = ['coveredLinePercent', 'coveredBranchPercent', 'coveredFunctionPercent'];
const kColumns = ['stmts %', 'line %', 'branch %', 'funcs %'];
const kColumnsKeys = [
'coveredStatementPercent', 'coveredLinePercent',
'coveredBranchPercent', 'coveredFunctionPercent',
];
const kSeparator = ' | ';

function buildFileTree(summary) {
Expand Down
5 changes: 5 additions & 0 deletions src/node_options.cc
Original file line number Diff line number Diff line change
Expand Up @@ -935,6 +935,11 @@ EnvironmentOptionsParser::EnvironmentOptionsParser() {
&EnvironmentOptions::test_coverage_lines,
kAllowedInEnvvar,
OptionNamespaces::kTestRunnerNamespace);
AddOption("--test-coverage-statements",
"the statement coverage minimum threshold",
&EnvironmentOptions::test_coverage_statements,
kAllowedInEnvvar,
OptionNamespaces::kTestRunnerNamespace);
AddOption("--test-isolation",
"configures the type of test isolation used in the test runner",
&EnvironmentOptions::test_isolation,
Expand Down
Loading