Bender aims to be a lightweight framework that executes its neural networks on the GPU and also allows to import models from other frameworks without adding big binaries. In the case of TensorFlow, you could use it to run code using Accelerate framework (CPU) but to include TensorFlow in your app you have to compile it as a static binary and it will add around 23 MB to your app (June 2017). Bender offers a lightweight solution which also uses the GPU for performance gain.
This guide includes the following points:
If you can visualize the graph generated by your TensorFlow model using TensorBoard then it will be much easier for to you to know what is happening.
You could use TensorFlow's freeze_graph function to have a ready-to-use version for Bender. However, we recommend using the tf-freeze utility from benderthon, which is generally easier to use. First you need to install it (pip install benderthon) and then you can freeze an existing checkpoint:
benderthon tf-freeze checkpoint_path.ckpt graph_with_weights.pb Output_Node_NameTo import a model saved in a Protobuf file you must add it to your Xcode project and load it like this:
// Set an url pointing to your model
let url = Bundle.main.url(forResource: "myGraph", withExtension: "pb")!
// Create the converter
let converter = TFConverter.default()
// Load it
let network = Network.load(url: url, converter: converter, inputSize: LayerSize(h: 256, w: 256, f: 3))TFConverter is the class responsible for converting a TF model to Bender. It will try to map nodes or groups of nodes in the TF graph to Bender layers. If it encounters unknown nodes then it will ignore them. This means that a graph might be disconnected if your TF model uses functions that are not implemented in Bender.
The default converter will make the following simplifications / optimizations:
- Remove subgraphs named with 'save', 'dropout', 'Regularizer' and 'Initializer' as these are training functions. There will certainly be more subgraphs added to this list in the future. This is implemented with
TFStripTrainingOpsandTFDeleteDropout. - Process Variables (
TFVariableProcessor): It leaves only constant variables (not randomly initialized). - Create FullyConnected (
TFDenseSubstitution): In TF, the fully connected layer is implemented as a MatMul with an BiasAdd (or Add). If these nodes appear in the graph then they will be substituted by a 'FullyConnected' node. - Remove Reshape (
TFReshapeOptimizer): Removes reshape nodes from the graph. We should investigate if in future releases we want to do something else in this case. - Add bias to a Convolution (
TFConvOptimizer).
Unsupported layers are ignored and their connections are not rewired. This means that if you have a layer that is not supported in your main execution graph then the graph will be separated in different parts and the conversion will fail. If you have unsupported layers in your graph which you want to ignore then you should add an optimizer similar to TFDeleteDropout to your converter. If you have a layer in your graph that is not supported but you need to execute it in Bender then you should add a custom layer implementing it and create a mapper that maps that operation to the layer you created.
In TensorFlow, the weights for Conv2D and most other layers are stored in [kernelHeight, kernelWidth, inputChannels, outputChannels] order but Metal requires a different order. This means that the weights need to be transposed. This can be done on the Python or on the Swift side.
For MPSCNNConvolution you need to pass [outputChannels, kernelHeight, kernelWidth, inputChannels] ordered weights, while the MPSCNNFullyConnected class takes weights in the order [outputChannels, sourceWidth, sourceHeight, inputChannels]. Bender includes helper functions to make this transpositions which are used when converting these layers. Have a look at HWIOtoOHWI for an example. If you want to create another layer which needs transposition of weights then you can implement your own in a similar way.
If the default optimizers do not fill your needs you might want to add custom optimizers. Also keep in mind that Bender does not support every function implemented in TensorFlow so it might be that you use not supported ops. In this case you can create a custom layer as explained below. We are open to receive Pull Requests implementing more layers.
To create an optimizer, create a class that conforms to TFOptimizer.
A TFOptimizer does just have one simple function:
func optimize(graph: TFGraph)After you create the optimizer, you have to add it to your TFConverter like this:
let converter = TFConverter.default(additionalOptimizers: [MyTFOptimizer()])It is common that there are nodes in a graph that are only important in training and we do not want them in the final execution graph. Sometimes we have to remove these nodes with an optimizer.
Sometimes TensorFlow creates subgraphs which are named like 'subgraph_name/.../...'. For example a dropout consists of a lot of nodes called like 'dropout/...'. These subgraphs can be easily removed by subclassing TFDeleteSubgraphOptimizer and setting its regex variable with a regular expression that matches you subgraph.
Have a look at TFDeleteOptimizers.swift for examples.
Other times, if you want to remove a single node then you can also create an optimizer that traverses the graph and calls either strip (which removes all the connections of the node without rewiring them) or removeFromGraph (which does the same but also rewires the incoming connections to the outgoing connections) on each node that you want to remove. After each optimization the nodes that have no connections will be removed from the graph.
Sometimes TensorFlow creates several nodes which we want to map to a single layer in Bender. An example of this is TFInstanceNormOptimizer which converts several nodes to an InstanceNorm. As there is no Instance Normalization operation in TensorFlow, each model might have a custom implementation. Bender includes an optimizer that converts the implementation as you can find it in the Fast Style transfer repository.
The idea of combining nodes is to create something a mapper can convert into a Bender layer. For that it can be helpful to add attributes to one of the nodes that you want to combine and remove the other nodes from the graph.
A mapper is function that converts one node from a TensorFlow graph to one layer in Bender.
In API you can find how to create custom layers. Therefore we will concentrate in creating mappers that convert a node as it comes from a TF graph into a Bender NetworkLayer.
A TFConverter has a dictionary that maps TF ops to TFMappers:
public var mappers = [String: TFMapper]()A TFMapper is just a function that creates a NetworkLayer given a TFNode:
public typealias TFMapper = (TFNode) -> NetworkLayer
So, if you want to add a mapper, then you have to add it to the converter's mappers variable.
Have a look at TFConverter+Mappers.swift to see examples of mappers.