Guide or docs for setting resource spec for KubeflowV2DagRunner?

Hi, there.

Is there any guide or best practice to set custom resource specs (CPUs, RAM, GCE node spec, …) for KubeflowV2DagRunner? I checked the related issue and protobuf specs.

It works, but I think that it is not a clean or good way to set the specs compared to the V1 Runner way, since it relies on undocumented JSON data. (In V1 runner, It supports pipeline_operator_funcs arg in KubeflowDagRunnerConfig.)

Thanks :slight_smile:


You can pass the Custom CPU and Memory in with_platform_config as shown in b15d592

For Example :

    bq_example_gen = big_query_example_gen_component.BigQueryExampleGen(
            .ResourceSpec(cpu_limit=5.0, memory_limit=10.0))
    deployment_config = pipeline_pb2.PipelineDeploymentConfig()

Thank you!