coremltools 8.0b1

For all the new features, find the updated documentation in the docs-guides

• New utilities coremltools.utils.MultiFunctionDescriptor() and coremltools.utils.save_multifunction , for creating an mlprogram with multiple functions in it, that can share weights. Updated the model loading API to load specific functions for prediction.
• Stateful Core ML models: updates to the converter to produce Core ML models with the State Type (new type introduced in iOS18/macOS15).
• coremltools.optimize
  • Updates to model representation (mlprogram) pertaining to compression:
    • Support compression with more granularities: blockwise quantization, grouped channel wise palettization
    • 4 bit weight quantization (in addition to 8 bit quantization that was already supported)
    • 3 bit palettization (in addition to 1,2,4,6,8 bit palettization that was already supported)
    • Support joint compression modes:
      • 8 bit Look-up-tables for palettization
      • ability to combine weight pruning and palettization
      • ability to combine weight pruning and quantization
  • API updates:
    • coremltools.optimize.coreml
      • Updated existing APIs to account for features mentioned above
      • Support joint compression by applying compression techniques on an already compressed model
      • A new API to support activation quantization using calibration data, which can be used to take a W16A16 Core ML model and produce a W8A8 model: ct.optimize.coreml.experimental.linear_quantize_activations
        • (to be upgraded from experimental to the official name space in a future release)
    • coremltools.optimize.torch
      • Updated existing APIs to account for features mentioned above
      • Added new APIs for data free compression (PostTrainingPalettizer , PostTrainingQuantizer
      • Added new APIs for calibration data based compression (SKMPalettizer for sensitive k-means palettization algorithm, layerwise_compression for GPTQ/sparseGPT quantization/pruning algorithm)
      • Updated the APIs + the coremltools.convert implementation, so that for converting torch models compressed with ct.optimize.torch , there is no longer a need to provide additional pass pipeline arguments.
• iOS18 / macOS15 ops
  • compression related ops: constexpr_blockwise_shift_scale, constexpr_lut_to_dense, constexpr_sparse_to_dense, etc
  • updates to the GRU op
  • PyTorch op scaled_dot_product_attention
• Experimental torch.export conversion support

import torch
import torchvision

import coremltools as ct

torch_model = torchvision.models.vit_b_16(weights="IMAGENET1K_V1")

x = torch.rand((1, 3, 224, 224))
example_inputs = (x,)
exported_program = torch.export.export(torch_model, example_inputs)

coreml_model = ct.convert(exported_program)

• Various other bug fixes, enhancements, clean ups and optimizations

Known Issues

• Conversion will fail when using certain palettization modes (e.g. int8 LUT, vector palettization) with torch models using ct.optimize.torch
• Some of the joint compression modes when used with the training time APIs in ct.optimize.torch will result in a torch model that is not correctly converted
• The post-training palettization config for mlpackage models (ct.optimize.coreml.``OpPalettizerConfig) does not yet have all the arguments that are supported in the cto.torch.palettization APIs (e.g. lut_dtype (to get int8 dtyped LUT), cluster_dim (to do vector palettization), enable_per_channel_scale (to apply per-channel-scale) etc).

Special thanks to our external contributors for this release: @teelrabbit @igeni @Cyanosite

coremltools 7.2

• New Features
  • Supports ExecuTorch 0.2 (see ExecuTorch doc for examples)
    • Core ML Partitioner: If a PyTorch model is partially supported with Core ML, then Core ML partitioner can determine the supported part and have ExecuTorch delegate to Core ML.
    • Core ML Quantizer: Quantize PyTorch models in Core ML favored scheme
• Enhancements
  • Improved Model Conversion Speed
  • Expanded Operation Translation Coverage
    • add torch.narrow
    • add torch.adaptive_avg_pool1d and torch.adaptive_max_pool1d
    • add torch.numpy_t (i.e. the numpy-style transpose operator .T)
    • enhance torch.clamp_min for integer data type
    • enhance torch.add for complex data type
    • enhance tf.math.top_k when k is variable

Thanks to our ExecuTorch partners and our open-source community: @KrassCodes @M-Quadra @teelrabbit @minimalic @alealv @ChinChangYang @pcuenca

coremltools 7.1

• New Features:

  • Supports Torch 2.1

    • Includes experimental support for torch.export API but limited to EDGE dialect.

    • Example usage:

      •  import torch 
         from torch.export import export
         from executorch.exir import to_edge
         
         import coremltools as ct
         
         example_args = (torch.randn(*size), )
         aten_dialect = export(AnyNNModule(), example_args)
         edge_dialect = to_edge(aten_dialect).exported_program()
         edge_dialect._dialect = "EDGE"
         
         mlmodel = ct.convert(edge_dialect)
        

• Enhancements:

  • API - ct.utils.make_pipeline - now allows specifying compute_units
  • New optimization passes:
    • Folds selective data movement ops like reshape, transpose into adjacent constant compressed weights
    • Casts int32 → int16 dtype for all intermediate tensors when compute precision is set to fp16
  • PyTorch op - multinomial - Adds lowering for it to CoreML
  • Type related refinements on Pad and Gather/Gather-like ops

• Bug Fixes:

  • Fixes coremltools build issue related to kmeans1d package
  • Minor fixes in lowering of PyTorch ops: masked_fill & randint

• Various other bug fixes, enhancements, clean ups and optimizations.

coremltools 7.0

• New submodule coremltools.optimize for model quantization and compression
  • coremltools.optimize.coreml for compressing coreml models, in a data free manner. coremltools.compresstion_utils.* APIs have been moved here
  • coremltools.optimize.torch for compressing torch model with training data and fine-tuning. The fine tuned torch model can then be converted using coremltools.convert
• The default neural network backend is now mlprogram for iOS15/macOS12. Previously calling coremltools.convert() without providing the convert_to or the minimum_deployment_target arguments, used the lowest deployment target (iOS11/macOS10.13) and the neuralnetwork backend. Now the conversion process will default to iOS15/macOS12 and the mlprogram backend. You can change this behavior by providing a minimum_deployment_target or convert_to value.
• Python 3.11 support.
• Support for new PyTorch ops: repeat_interleave, unflatten, col2im, view_as_real, rand, logical_not, fliplr, quantized_matmul, randn, randn_like, scaled_dot_product_attention, stft, tile
• pass_pipeline parameter has been added to coremltools.convert to allow controls over which optimizations are performed.
• MLModel batch prediction support.
• Support for converting statically quantized PyTorch models.
• Prediction from compiled model (.modelc files). Get compiled model files from an MLModel instance. Python API to explicitly compile a model.
• Faster weight palletization for large tensors.
• New utility method for getting weight metadata: coremltools.optimize.coreml.get_weights_metadata. This information can be used to customize optimization across ops when using coremltools.optimize.coreml APIs.
• New and updated MIL ops for iOS17/macOS14/watchOS10/tvOS17
• coremltools.compression_utils is deprecated.
• Changes default I/O type for Neural Networks to FP16 for iOS16/macOS13 or later when mlprogram backend is used.
• Changes upper input range behavior when backend is mlprogram:
  • If RangeDim is used and no upper-bound is set (with a positive number), an exception will be raised.
  • If the user does not use the inputs parameter but there are undetermined dim in input shape (for example, TF with "None" in input placeholder), it will be sanitized to a finite number (default_size + 1) and raise a warning.
• Various other bug fixes, enhancements, clean ups and optimizations.

Special thanks to our external contributors for this release: @fukatani , @pcuenca , @KWiecko , @comeweber , @sercand , @mlaves, @cclauss, @smpanaro , @nikalra, @jszaday

coremltools 7.0b2

• The default neural network backend is now mlprogram for iOS15/macOS12. Previously calling coremltools.convert() without providing the convert_to or the minimum_deployment_target arguments, used the lowest deployment target (iOS11/macOS10.13) and the neuralnetwork backend. Now the conversion process will default to iOS15/macOS12 and the mlprogram backend. You can change this behavior by providing a minimum_deployment_target or convert_to value.
• Changes default I/O type for Neural Networks to FP16 for iOS16/macOS13 or later when mlprogram backend is used.
• Changes upper input range behavior when backend is mlprogram:
  • If RangeDim is used and no upper-bound is set (with a positive number), an exception will be raised.
  • If the user does not use the inputs parameter but there are undetermined dim in input shape (for example, TF with "None" in input placeholder), it will be sanitized to a finite number (default_size + 1) and raise a warning.
• New utility method for getting weight metadata: coremltools.optimize.coreml.get_weights_metadata. This information can be used to customize optimization across ops when using coremltools.optimize.coreml APIs.
• Support for new PyTorch ops: repeat_interleave and unflatten.
• New and updated iOS17/macOS14 ops: batch_norm, conv, conv_transpose, expand_dims, gru, instance_norm, inverse, l2_norm, layer_norm, linear, local_response_norm, log, lstm, matmul, reshape_like, resample, resize, reverse, reverse_sequence, rnn, rsqrt, slice_by_index, slice_by_size, sliding_windows, squeeze, transpose.
• Various other bug fixes, enhancements, clean ups and optimizations.

Special thanks to our external contributors for this release: @fukatani, @pcuenca, @KWiecko, @comeweber and @sercand

coremltools 7.0b1

• New submodule coremltools.optimize for model quantization and compression
  • coremltools.optimize.coreml for compressing coreml models, in a data free manner. coremltools.compresstion_utils.* APIs have been moved here
  • coremltools.optimize.torch for compressing torch model with training data and fine-tuning. The fine tuned torch model can then be converted using coremltools.convert
• Updated MIL ops for iOS17/macOS14/watchOS10/tvOS17
• pass_pipeline parameter has been added to coremltools.convert to allow controls over which optimizations are performed.
• Python 3.11 support.
• MLModel batch prediction support.
• Support for converting statically quantized PyTorch models
• New Torch layer support: randn, randn_like, scaled_dot_product_attention, stft, tile
• Faster weight palletization for large tensors.
• coremltools.models.ml_program.compression_utils is deprecated.
• Various other bug fixes, enhancements, clean ups and optimizations.

Core ML tools 7.0 guide: https://coremltools.readme.io/v7.0/

Special thanks to our external contributors for this release: @fukatani, @pcuenca, @mlaves, @cclauss, @smpanaro, @nikalra, @jszaday

coremltools 6.3

Core ML Tools 6.3 Release Note

• Torch 2.0 Support
• TensorFlow 2.12.0 Support
• Remove Python 3.6 support
• Functionality for controling graph passes/optimizations, see the pass_pipeline parameter to coremltools.convert.
• A utility function for easily creating pipeline, see: utils.make_pipeline.
• A debug utility function for extracting submodels, see: converters.mil.debugging_utils.extract_submodel
• Various other bug fixes, enhancements, clean ups and optimizations.

Special thanks to our external contributors for this release: @fukatani, @nikalra and @kevin-keraudren.

coremltools 6.2

Core ML Tools 6.2 Release Note

• Support new PyTorch version: torch==1.13.1 and torchvision==0.14.1.
• New ops support:
  • New PyTorch ops support: 1-D and N-D FFT / RFFT / IFFT / IRFFT in torch.fft, torchvision.ops.nms, torch.atan2, torch.bitwise_and, torch.numel,
  • New TensorFlow ops support: FFT / RFFT / IFFT / IRFFT in tf.signal, tf.tensor_scatter_nd_add.
• Existing ops improvements:
  • Supports int input for clamp op.
  • Supports dynamic topk (k not determined during compile time).
  • Supports padding='valid' in PyTorch convolution.
  • Supports PyTorch Adaptive Pooling.
• Supports numpy v1.24.0 (#1718)
• Add int8 affine quantization for the compression_utils.
• Various other bug fixes, optimizations and improvements.

Special thanks to our external contributors for this release: @fukatani, @ChinChangYang, @danvargg, @bhushan23 and @cjblocker.

coremltools 6.1

• Support for TensorFlow 2.10.
• New PyTorch ops supported: baddbmm, glu, hstack, remainder, weight_norm, hann_window, randint, cross, trace, and reshape_as.
• Avoid root logger and use the coremltools logger instead.
• Support dynamic input shapes for PyTorch repeat and expand op.
• Enhance translation of torch where op with only one input.
• Add support for PyTorch einsum equation: 'bhcq,bhck→bhqk’.
• Optimization graph pass improvement
  • 3D convolution batchnorm fusion
  • Consecutive relu fusion
  • Noop elimination
• Actively catch the tensor which has rank >= 6 and error out
• Various other bug fixes, optimizations and improvements.

Special thanks to our external contributors for this release: @fukatani, @piraka9011, @giorgiop, @hollance, @SangamSwadiK, @RobertRiachi, @waylybaye, @GaganNarula, and @sunnypurewal.

coremltools 6.0

• MLProgram compression: affine quantization, palettize, sparsify. See coremltools.compression_utils
• Python 3.10 support.
• Support for latest scikit-learn version (1.1.2).
• Support for latest PyTorch version (1.12.1).
• Support for TensorFlow 2.8.
• Support for options to specify input and output data types, for both images and multiarrays
  • Update coremltools python bindings to work with GRAYSCALE_FLOAT16 image datatype of CoreML
  • New options to set input and output types to multi array of type float16, grayscale image of type float16 and set output type as images, similar to the coremltools.ImageType used with inputs.
• New compute unit enum type: CPU_AND_NE to select the model runtime to the Neural engine and CPU.
• Support for several new TensorFlow and PyTorch ops.
• Changes to opset (available from iOS16, macOS13)
  • New MIL ops: full_like, resample, reshape_like, pixel_unshuffle, topk
  • Existing MIL ops with new functionality: crop_resize, gather, gather_nd, topk, upsample_bilinear.
• API Breaking Changes:
  • Do not assume source prediction column is “predictions”, fixes #58.
  • Remove useCPUOnly parameter from coremltools.convert and coremltools.models.MLModel. Use coremltools.ComputeUnit instead.
  • Remove ONNX support.
  • Remove multi-backend Keras support.
• Various other bug fixes, optimizations and improvements.