rajatmonga

Agree, that is a big problem that we are working hard to solve. It isn't solved in this release, but it is high up on our task list.

1. The code is standard C/C++ with minimal dependencies so it should be buildable on even non-standard platforms. Linux is easy.

2. The interpreter is more optimized for being low overhead and the kernels are better optimized especially for ARM CPUs currently. While model performance varies by model - we have seen significant improvements on most models going from TensorFlow to TensorFlow Lite. We'll share benchmarks soon.

The current examples talk about Android/iPhone, however the core runtime is pretty lightweight with the goal of supporting all kinds of embedded products.

Do let us know if you build/run on other platforms.

With TensorFlow and TF Lite we are looking to provide a great experience across all platforms, and are exploring ways to provide a simpler experience with good acceleration on iOS as well.

XLA for AOT is useful for cases when you know exactly what architecture you are shipping to, and are ok updating the code whenever the model changes.

TF Lite addresses the segment where you need more flexibility

- you ship single app to many types of devices

- would like to update the model independent of the code itself e.g. no change to Android APK, and update the model over the wire.

Even with this generality, TF Lite is still quite fast and lightweight as that was the focus building it up.

TF Lite supports Android NN API that allows each phone to accelerate these models leveraging the custom accelerator on the phone.

A few tradeoffs we had to make:

- As mentioned below - flatbuffers makes the startup time faster while trading off some flexibility

- Smaller code size means trading off dependency on some libraries and broader support vs writing more things from scratch more focused on the user cases people care about

TensorFlow Lite is TensorFlow’s lightweight solution for mobile and embedded devices! TensorFlow has always run on many platforms, from racks of servers to tiny devices, but as the adoption of machine learning models has grown over the last few years, so has the need to deploy them on mobile and embedded devices. TensorFlow Lite enables low-latency inference of on-device machine learning models.

Looking forward to your feedback as you try it out.