test: add unit tests for EmbeddingModel, GAN, metrics, and label processors#956
Open
sacredvoid wants to merge 1 commit intosunlabuiuc:masterfrom
Open
test: add unit tests for EmbeddingModel, GAN, metrics, and label processors#956sacredvoid wants to merge 1 commit intosunlabuiuc:masterfrom
sacredvoid wants to merge 1 commit intosunlabuiuc:masterfrom
Conversation
…essors Add 6 new test files with 73 test cases covering previously untested modules. All tests use synthetic data, complete in under 5 seconds, and follow the established test patterns. Models: - test_embedding_model.py: EmbeddingModel with 5 processor types (sequence, tensor, multi_hot, nested_sequence, mixed) - test_gan.py: GAN discriminator/generator for 32x32, 64x64, 128x128 Metrics: - test_binary_metrics.py: binary_metrics_fn (9 metrics + calibration) - test_multiclass_metrics.py: multiclass_metrics_fn (ROC-AUC, F1, Jaccard, ECE, hits@n, mean_rank) - test_regression_metrics.py: regression_metrics_fn (MSE, MAE, KL) Processors: - test_label_processor.py: BinaryLabelProcessor, MultiClassLabelProcessor Note: test_binary_metrics includes a skipped test documenting a bug in ece_confidence_binary (calibration.py:150) where 1D arrays are indexed as 2D. Ref sunlabuiuc#425
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add 6 new test files with 73 test cases covering modules that previously had no unit tests:
EmbeddingModel,GAN,binary_metrics_fn,multiclass_metrics_fn,regression_metrics_fn,BinaryLabelProcessor, andMultiClassLabelProcessor.All tests use synthetic data, complete in under 5 seconds, and follow the established patterns from
test_transformer.py/test_retain.py.test_embedding_model.py (14 tests)
test_initializationtest_embedding_layers_are_correct_typetest_forward_output_shapestest_forward_with_output_masktest_gradients_flowtest_mixed_forwardtest_custom_embedding_dimtest_gan.py (14 tests)
test_initializationtest_discriminator_output_shapetest_discriminator_output_rangetest_generate_fake_shapetest_generate_fake_pixel_rangetest_sampling_shapetest_discriminator_backward/test_generator_backwardtest_end_to_end/test_multichannel_end_to_endtest_binary_metrics.py (10 tests)
Tests
binary_metrics_fnwith all 9 supported metrics (pr_auc, roc_auc, accuracy, balanced_accuracy, f1, precision, recall, cohen_kappa, jaccard), plus default behavior, custom thresholds, value range validation, perfect predictions, and error handling. One test is skipped documenting a bug inece_confidence_binary(calibration.py:150) where 1D arrays are indexed as 2D.test_multiclass_metrics.py (11 tests)
Tests
multiclass_metrics_fnwith ROC-AUC variants (macro/weighted, ovo/ovr), F1 variants (micro/macro/weighted), Jaccard variants, calibration metrics (ECE, brier_top1, cwECEt), cohen_kappa, hits@n, and mean_rank.test_regression_metrics.py (9 tests)
Tests
regression_metrics_fnwith MSE, MAE, KL divergence, perfect reconstruction (zero error), shape mismatch error handling, and 2D array flattening.test_label_processor.py (15 tests)
Tests
BinaryLabelProcessor(fitting with int/bool/string labels, non-binary raises error, process output, size, schema) andMultiClassLabelProcessor(fitting, correct index mapping, size, schema).Pattern
Follows the established test patterns:
create_sample_dataset()or small numpy arraystorch.manual_seed(42)/np.random.seed(42)for determinismunittest.TestCasestructureHow to test
Ref #425