Tensorflow gpu serving without docker on "windows"

Leema_K · August 20, 2021, 3:12pm

Bhack · August 20, 2021, 4:16pm

WSL2 Is stable now so I think the stackoverflow topic is outdated.

Leema_K · August 20, 2021, 4:32pm

What do you mean by stable? Are you saying we can do the tf gpu serving without registering in windows insider program?

Bhack · August 20, 2021, 4:48pm

Yes:

Leema_K · August 20, 2021, 5:09pm

Thanks, Stefano. I think I have not made myself clear. I have installed WSL2. But for tf serving, the documentations say the best way to do is using docker image. For that, I have to install nvidia-docker. But again for installing nvidia-docker in WSL2, this documentation says the system should be registered in windows Insider program, which my org doesn’t allow me to do. Are there any other ways to do tf gpu serving without docker.

Bhack · August 20, 2021, 6:55pm

For that one yes, on Windows 10 you need insider or you need to use Windows 11.

You could try to build It from sources on Windows adapting the Docker source build guide but It is not offcially supported:

github.com

tensorflow/serving/blob/master/tensorflow_serving/g3doc/setup.md#building-from-source

# Installation

## Installing ModelServer

### Installing using Docker

The easiest and most straight-forward way of using TensorFlow Serving is with
[Docker images](docker.md). We highly recommend this route unless you have
specific needs that are not addressed by running in a container.

TIP: This is also the easiest way to get TensorFlow Serving working with [GPU
support](docker.md#serving-with-docker-using-your-gpu).

### Installing using APT

#### Available binaries

The TensorFlow Serving ModelServer binary is available in two variants:

**tensorflow-model-server**: Fully optimized server that uses some platform

This file has been truncated. show original

Leema_K · August 21, 2021, 3:33pm

This helps! Thanks a lot, Stefano!

Bhack · August 21, 2021, 3:58pm

If you can complete the build please leave a comment with your solution also in: