Anyone implemented TFX in AWS?


I was wondering if it possible/how easy it would be to implement a TFX pipeline (on a real dataset, with 100+ GB dataset, not a tutorial with a small dataset) in AWS?

For the orchestration, I might use Kubeflow. But I suppose, the major issue would be setting up a proper scalable runner for the Apache Beam. I am thinking of using Apache Flink for that.

Anyone with experience doing it? How would you go about putting a TF in production in AWS in general when you need to train the model on a regular basis on new data, do you write the pipeline from scratch or use some tool?

Thank You,
Gorjan Todorovski

1 Like

Generally I suggest you to select tags and category from the menu on a new thread cause specialized technical team members could be subscribed only to a tag subset (e.g. in this case I suggest tfx tag)

1 Like

Idon’t know how to add tags and even don’t see an option to edit the post.

1 Like

Do you see these options with your account

A comprehensive guide to Discourse tags - faq - Discourse Meta ?

1 Like