How do you want to run your production ML infrastructure? Do you prefer cloud or on-prem deployments? For cloud, do you prefer managed or self-managed?
2 Likes
This is a great question for a thorough discussion.
I usually prefer the cloud and decide whether or not to use a managed service based on several considerations. First, if the service is supposed to run infrequently at unpredicted times (usually at early stages of an application), I prefer managed services as they give me a predictable cost overview. It also helps me avoid issues caused by cold start in the serverless. I simply dockerize my services and docker-compose-up them with different .env files in dev and prod environments. This seems a bit like hacky but yields a great productivity at early stages.
As my application starts to get attension and have a more regular usage pattern, I start to migrate / upgrade Docker containers to managed services one by one. Finally, I try to implement MLOps best practices for CI/CD/CT.
This workflow helps me remain experimental, fast-prototyping and cost-effective at early stages and then become gradually more robust, tested, scalable and manageable.
I love to hear from others’ experiences in this.
3 Likes
It is very important for us to use cloud-managed services. We mainly use AI Platform Pipelines (Kubeflow Pipelines) for production.
In most cases, we start prototyping in a local notebook environment. After the prototyping, we design and develop an ML pipeline on AI Platform Pipelines. This works for us because we built relatively simple batch pipelines. If our task is a more complex one or needs online evaluation, we might have took a different way.
1 Like