After releasing its deep learning or machine learning library TensorFlow as open source software a few months ago, Google has now followed up by open sourcing TensorFlow Serving, which the company says makes it easier to take models into production.
Noah Fiedel, Google software engineer, writes on his blog: “TensorFlow Serving makes the process of taking a model into production easier and faster.
“It allows you to safely deploy new models and run experiments while keeping the same server architecture and APIs.”
TensorFlow Serving is written in C++ and supports Linux. Fidel says the software requires minimal overhead. “In our benchmarks we recoded approximately 100,000 queries per second per core on a 16 vCPU Intel Xeon E5 2.6 GHz machine, excluding gRPC and the TensorFlow inference processing time.”