RecipeTestRunner
Recipe-based test runner.
Discovers recipes with gold standards and runs them as regression tests. This replaces the complex TestCase approach with a simple:
run recipe → compare to gold → pass/fail on Dice
Classes
- class RecipeTestResult
Result of a single recipe test.
- class RecipeTestSuiteResult
Result of running all recipe tests.
Methods:
- passed()
- passed_count()
- total_count()
- class RecipeTestRunner
Run recipes as regression tests.
Discovers all recipes that have a gold_standard field and runs them, comparing output to the gold standard segmentation.
- Usage:
runner = RecipeTestRunner() result = runner.run_all() print(f”Passed: {result.passed_count}/{result.total_count}”)
Methods:
- __init__()
Initialize test runner.
- discover_recipes()
Find all recipes with gold standards.
- run_recipe_test()
Run a single recipe test.
- run_all()
Run all recipe tests.
Functions
- passed()
- passed_count()
- total_count()
- discover_recipes()
Find all recipes with gold standards.
- Returns:
List of (recipe_path, gold_standard_name) tuples.
- run_recipe_test()
Run a single recipe test.
- Args:
recipe_path: Path to recipe file. gold_name: Name of gold standard to compare against. test_run_folder: Optional test run folder for output.
- Returns:
RecipeTestResult with pass/fail and metrics.
- run_all()
Run all recipe tests.
- Returns:
RecipeTestSuiteResult with all test results.
- run_recipe_tests()
Convenience function to run all recipe tests.
- Returns:
RecipeTestSuiteResult with all test results.