Test Suite Execution

Test suite execution validates the transpiled Rust code by running cargo test in the output directory. This catches regressions in both transpiler-generated tests and manually written tests.

How It Works

The ValidationStage reads the output directory from batuta.toml and runs the test suite:

# Batuta runs this internally
cd ./rust-output && cargo test

Test output is captured and parsed. A non-zero exit code marks the validation as failed.

Test Sources

Transpiled projects can contain tests from multiple origins:

Source	Description
Transpiler-generated	Depyler/Decy/Bashrs generate test stubs from the original code
Manually written	Developer-added tests for edge cases
Property-based	Generated by `proptest` for invariant checking
Migrated	Original test suite adapted to Rust

Property-Based Testing

For numerical code (common in ML pipelines), property-based testing with proptest provides stronger guarantees than example-based tests:

#![allow(unused)]
fn main() {
use proptest::prelude::*;

proptest! {
    #[test]
    fn vector_add_commutative(
        a in prop::collection::vec(-1e6f32..1e6, 1..1000),
        b in prop::collection::vec(-1e6f32..1e6, 1..1000),
    ) {
        let len = a.len().min(b.len());
        let a = &a[..len];
        let b = &b[..len];
        // a + b == b + a
        let result1 = vector_add(a, b);
        let result2 = vector_add(b, a);
        assert_eq!(result1, result2);
    }
}
}

Coverage Tracking

Batuta integrates with cargo llvm-cov to track test coverage of the transpiled code:

# Run tests with coverage
batuta validate --run-original-tests --coverage

# Coverage report
batuta validate --coverage-report

Coverage: 87.3% (target: 95%)
  src/main.rs     92.1%
  src/utils.rs    84.5%
  src/parser.rs   79.2%  <-- below target

CLI Usage

# Run transpiled test suite
batuta validate --run-original-tests

# Run with verbose test output
batuta validate --run-original-tests --verbose

# Run specific test
batuta validate --run-original-tests --test test_name

# Run with nextest for parallel execution
batuta validate --run-original-tests --nextest

Test Failure Handling

Test failures are recorded in the ValidationResult with full output. The validation phase is marked as failed, blocking Phase 5 (Deployment) until all tests pass.

Navigate: Table of Contents

Keyboard shortcuts