TensorFlow Model Analysis 0.52.0

Major Features and Improvements

• Adds official support for Python 3.12 and 3.13.
• Drops support for Python 3.9.
• Depends on tensorflow>=2.21.0,<2.22.0.
• Depends on protobuf>=6.31.1,<7.0.0.
• Depends on pyarrow>14.
• Updates the minimum Bazel version required to build TFMA to 7.4.1.

Bug fixes and other Changes

• Pickling Stability Architecture:
  • Refactored core test modules (rouge_test.py, stats_test.py, model_util_test.py, confidence_intervals_util_test.py, metrics_plots_and_validations_evaluator_test.py) to use a new class-based matcher architecture.
  • Replaced nested closures with module-level classes (e.g., CheckResult, CheckResultMean) to ensure full serializability for PrismRunner on Python 3.13.
  • Removed self (test instance) capture in Beam matchers to resolve RuntimeError: Unable to pickle fn during distributed execution.
  • Enabled --no_save_main_session for all Beam pipelines in the test suite to prevent unintentional serialization of the main session and shared resources.
• NumPy 2.0 & Python 3.13 Compatibility:
  • Standardized on safe scalar extraction by replacing float(ndarray) with .item() or metric_util.safe_to_scalar in aggregation, attributions, calibration, flip metrics, and NDCG modules to resolve TypeError in Beam pipelines.
  • Fixed a batching bug in flip_metrics.py to correctly process all examples in a Beam batch.
  • Implemented robust, warning-free division in AUC, PR AUC, and confusion matrix calculations using np.divide with where clauses to prevent RuntimeWarning.
  • Fixed TypeError in poisson_bootstrap.py by removing redundant size argument from poisson(1, 1) to return a scalar.
  • Implemented kind='stable' sort in metric_util.top_k_indices for deterministic tie-breaking.
• Bug Fixes and Functional Corrections:
  • Fixed a critical regression in metric_util.py where SubKey(k=k) incorrectly selected the first prediction instead of the requested k-th largest prediction.
  • Fixed UnparsedFlagAccessError in ModelSignaturesDoFn tests by removing direct absl.flags access in pickling-sensitive contexts.
  • Removed obsolete @unittest.expectedFailure decorators from tests that are now passing in the stabilized environment.
  • Fixed various indentation and syntax errors in utility tests.
  • Improved virtual environment relocation strategy to resolve Bazel sandbox access issues for numpy and other C-extension headers.
  • Fixed false_omission_rate in binary_confusion_matrices.py to return NaN when undefined, resolving proto mismatches in confusion_matrix_plot_test.py and score_distribution_plot_test.py.
  • Fixed NotFoundError in model_eval_lib_test.py by ensuring temporary directories exist before writing files using tf.io.gfile.makedirs.
  • Added missing numpy imports in Beam-based modules to fix NameError regressions.

Breaking Changes

• Python 3.9 is no longer supported. The minimum supported Python version is now 3.10.

Deprecations

• N/A

TensorFlow Model Analysis 0.48.0

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Depends on tensorflow>=2.17,<2.18.
• Depends on protobuf>=4.25.2,<5 for Python 3.11 and on
  protobuf>=4.21.6,<6.0.0 for 3.9 and 3.10.
• Depends on apache-beam[gcp]>=2.53.0,<3 for Python 3.11 and on
  apache-beam[gcp]>=2.50.0,<2.51 for 3.9 and 3.10.
• macOS wheel publishing is temporarily paused due to missing ARM64 support.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.47.1

Version 0.47.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Removing addons from init.py as it's deprecated with Eval Saved Model.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.47.0

Major Features and Improvements

• Adds False{Negative,Positive}FeatureSampler metrics.
• Adds RecallAtFalsePositiveRate metrics.
• Adds 'NegToNegFlipRate', 'NegToPosFlipRate', 'PosToNegFlipRate',
  'PosToPosFlipRate', and 'SymmetricFlipRate' metrics.
• Graduates dataframe module to core library as tfma.dataframe.

Bug fixes and other Changes

• Adds support for absolute change threshold validation on native diff metrics
  like flip rate metrics (e.g. SymmetricFlipRate) and
  ModelCosineSimilarity.
• Fixes a bug by ensuring feature values are always numpy arrays.
• Modifies a ROUGE Test to be compatible with NumPy v2.0.1.
• Remove keras_util_test.py which is based on estimator models.
• Remove dependency on eval_saved_model encodings.
• Downloads punkt_tab in Rouge metric.
• Depends on tensorflow 2.16
• Relax dependency on Protobuf to include version 5.x

Breaking Changes

• Removing legacy_predict_extractor in model_eval_lib.py.
• Removing post_export_metrics support in model_eval_lib.py.
• Removing eval saved model related API in
  metrics_plots_and_validations_evaluator.py
• Removing legacy_metrics_and_plots_evaluator in model_eval_lib.py.
• Removes legacy_metrics_and_plots_evaluator from the public API of TFMA
  evaluator.
• Removing eval_saved_model related API in model_util.py, estimator related
  functions are no longer supported.
• Removing legacy_metrics_and_plots_evaluator in TFMA OSS.
• Removing legacy_aggregate in TFMA.
• Remove legacy_query_based_metrics_evaluator.py and its test.
• Remove model_agnostic_eval, export, exporter, and post_export_metrics which are based on estimators.
• Remove eval_saved_model_util.py and its test.
• Remove contrib model_eval_lib and export and their tests.
• Remove all eval_saved_model files.

Deprecations

• Migrate common utils in eval_saved_model testutils to utils/test_util.py.
  This enables further deprecation of eval saved model.
• Deprecate legacy estimator related tests in predictions_extractor_test.py

TensorFlow Model Analysis 0.46.0

Major Features and Improvements

• Removes the metrics modules from experimental now that it is migrated to
  py-ml-metrics package.
• Adds Constituent Flip Rate Metrics: SymmetricFlipRate, NegToNegFlipRate,
  NegToPosFlipRate, PosToNegFlipRate, PosToPosFlipRate.
• Depend on tensorflow-estimator package explicitly.

Bug fixes and other Changes

• Fix the bug about batching unsized numpy arrays.

Breaking Changes

• Removes attrs requirement.
• Consolidate Matrix definition for semantic segmentation confusion matrix
  metrics.
• Provide AggregateFn and interface and default call impl to adapt TFMA
  metrics combiner for in-process call.
• Move Mean metrics from experimental to metrics.
• Fix the bug of size estimator failure.
• Depends on tensorflow>=2.15.0,<2.16.
• Fix the failure in testMeanAttributions.
• Fix the input type mismatch in metric_specs_tests between bool and None.
• Fix the failure in the slice test due to beam type hints check.
• Fix the failure in metric_specs test, all TFMA deps on keras are
  keras 2.
• Depends on apache-beam[gcp]>=2.53.0,<3 for Python 3.11 and on
  apache-beam[gcp]>=2.47.0,<3 for 3.9 and 3.10.
• Depends on protobuf>=4.25.2,<5 for Python 3.11 and on protobuf>3.20.3,<5
  for 3.9 and 3.10.
• Update the minimum Bazel version required to build TFMA to 6.1.0
• Refactors BooleanFlipRates computations to a combiner (flip_counts) and a
  DerivedMetricComputation (flip_rates).

Deprecations

• Deprecated python 3.8 support.

TensorFlow Model Analysis 0.45.0

Major Features and Improvements

• Add F1, False positive rate, and Accuracy into the confusion matrix plot.
• Add support for setting top_k and class_id at the same time for confusion
  matrix metrics.
• Add the false positive for semantic segmentation metrics.
• Add Mean Metric (experimental) which calculates the mean of any feature. *.
  Adds support of output_keypath to ModelSignatureDoFn to explicitly set a
  chain of output keys in the multi-level dict (extracts). Adds output_keypath
  to common prediction extractors.

Bug fixes and other Changes

• Fix the bug that SetMatchRecall is always 1 when top_k is set.
• Add 'tfma_eval' model_type in model_specs as the identifier for
  eval_saved_model, allowing signature='eval' to now be used with other model
  types.
• Add "materialized_prediction" model type to allow users bypassing model
  inference explicitly.
• Depends on pyarrow>=10,<11.
• Depends on apache-beam>=2.47,<3.
• Depends on numpy>=1.23.0.
• Depends on tensorflow>=2.13.0,<3.

Breaking Changes

• Depend on PIL for image related metrics.
• Separate extract_key from signature names in ModelSignaturesDoFn.

Deprecations

• N/A

TensorFlow Model Analysis 0.44.0

Major Features and Improvements

• Add BinaryCrossEntropy and CategoricalCrossEntropy.
• Add MeanAbsolutePercentageError and MeanSquaredLogarithmicError
• Add SetMatchPrecision and SetMatchRecall

Bug fixes and other Changes

• Fix for jupiter notebook
• Fix element dimension inconsistency when some of the extracts have missing
  key.
• Add public visibility to the servo beam extractor.
• Fix for bug where binary_confusion_matrices with different class_weights are
  considered identical and deduplicated.
• Fixes bug where positive / negative axes labels are reversed in prediction
  distribution plot.
• Depends on numpy~=1.22.0.
• Modify ExampleCount to not depend on labels and predictions.
• Add class_id info into sub_key in metric_key for object detection confusion
  matrix metrics.
• Add class_id info into sub_key in plot_key for object detection confusion
  matrix plot.
• Fix a bug that auto_pivot dropped nan when deciding which columns are
  multivalent for pivoting.
• Depends on tensorflow>=2.12.0,<2.13.
• Depends on protobuf>=3.20.3,<5.
• Depends on tfx-bsl>=1.13.0,<1.14.0.
• Depends on tensorflow-metadata>=1.13.1,<1.14.0.

Breaking Changes

• N/A

Deprecations

Deprecated python3.7 support.

TensorFlow Model Analysis 0.43.0

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Depends on tensorflow>=2.11,<3
• Depends on tfx-bsl>=1.2.0,<1.13.0.
• Depends on tensorflow-metadata>=1.12.0,<1.13.0.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.42.0

Major Features and Improvements

• This is the last version that supports TensorFlow 1.15.x. TF 1.15.x support
  will be removed in the next version. Please check the
  TF2 migration guide to migrate
  to TF2.
• Add BooleanFlipRate metric for comparing thresholded predictions between
  multiple models.
• Add CounterfactualPredictionsExtractor for computing predictions on modified
  inputs.

Bug fixes and other Changes

• Add support for parsing the Predict API prediction log output to the
  experimental TFX-BSL PredictionsExtractor implementation.

• Add support for parsing the Classification API prediction log output to the
  experimental TFX-BSL PredictionsExtractor implementation.

• Update remaining predictions_extractor_test.py tests to cover
  PredictionsExtractorOSS. Fixes a pytype bug related to multi tensor output.

• Depends on tensorflow>=1.15.5,<2 or tensorflow>=2.10,<3

• Apply changes in the latest Chrome browser

• Add InferneceInterface to experimental PredictionsExtractor implementation.

• Stop returning empty example_ids metric from binary_confusion_matrices
  derived computations when example_id_key is not set but use_histrogam is
  true.

• Add transformed features lookup for NDCG metrics query key and gain key.

• Deprecate BoundedValue and TDistribution in ConfusionMatrixAtThresholds.

• Fix a bug that dataframe auto_pivot fails if there is only Overall slice.

• Use SavedModel PB to determine default signature instead of loading the
  model.

• Reduce clutter in the multi-index columns and index in the experimental
  dataframe auto_pivot util.

Breaking Changes

• N/A

Deprecations

• N/A

Version 0.41.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from
  tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A

TensorFlow Model Analysis 0.41.1

Major Features and Improvements

• N/A

Bug fixes and other Changes

• Move the version to top of init.py since the original "from tensorflow_model_analysis.sdk import *" will not import private symbol.

Breaking Changes

• N/A

Deprecations

• N/A