I have a question about TFX KubeflowDagRunner.
Currently, in TFX, LocalDagRunner and BeamDagRunner inherit IrBasedRunner which is located in
tfx.orchestration.portable.tfx_runner, so we can serialize the desired pipelines as IRs and run them with only UDFs (i.e. run_fn or preprocess_fn of Trainer and Transform components) with
But I can’t do the same thing with the KubeflowDagRunner (or KubeflowV2DagRunner), because they don’t inherit IrBasedRunner. I checked KubeflowDagRunner is using TFX IR internally.
So my question is, is there a plan to update existing pipeline runners (especially, KubeflowDagRunner) to inherit
Always thanks for developing and maintaining this great project
- Pipeline IR protobuf spec: tfx/pipeline.proto at master · tensorflow/tfx · GitHub
- TFX IR RFC: community/20200705-tfx-ir.md at master · tensorflow/community · GitHub
- RFC status in TFX repo: tfx/RFCs.md at master · tensorflow/tfx · GitHub
- I checked the status of RFC in the above two links