Expectation
Use the tfdf.builder.CARTBuilder to build a decision tree structure and train it with the literal dataset, and optimize the tree structure per the performance.
The process is like manually replicate the training process using tfdf.keras.CartModel but the benefit is that I can adjust the tree structure per needs, not only focusing on the model performance, which will be helpful if intuitive rules are needed.
Sample code
I tried to use the tfdf.builder.CARTBuilder to build the structure and fit/predict but the results are not as expected as the fitting process does not lead to change of the prediction of leaves.
Below are some sample code with a sample dataset running in Colab
import tensorflow_decision_forests as tfdf
import os
import numpy as np
import pandas as pd
import tensorflow as tf
import matplotlib.pyplot as plt
import math
import collections
# Download the dataset
!wget -q https://storage.googleapis.com/download.tensorflow.org/data/palmer_penguins/penguins.csv -O /tmp/penguins.csv
# Load a dataset into a Pandas Dataframe.
dataset_df = pd.read_csv("/tmp/penguins.csv")
model_trial_idx = 10
# Create the model builder
model_trial_idx += 1
model_path = f"/tmp/manual_model/{model_trial_idx}"
builder = tfdf.builder.CARTBuilder(
path=model_path,
objective=tfdf.py_tree.objective.ClassificationObjective(
label="species", classes=["Adelie", "Non-Adelie"]))
# Create some alias
Tree = tfdf.py_tree.tree.Tree
SimpleColumnSpec = tfdf.py_tree.dataspec.SimpleColumnSpec
ColumnType = tfdf.py_tree.dataspec.ColumnType
# Nodes
NonLeafNode = tfdf.py_tree.node.NonLeafNode
LeafNode = tfdf.py_tree.node.LeafNode
# Conditions
NumericalHigherThanCondition = tfdf.py_tree.condition.NumericalHigherThanCondition
CategoricalIsInCondition = tfdf.py_tree.condition.CategoricalIsInCondition
# Leaf values
ProbabilityValue = tfdf.py_tree.value.ProbabilityValue
builder.add_tree(
Tree(
NonLeafNode(
condition=NumericalHigherThanCondition(
feature=SimpleColumnSpec(name="bill_length_mm", type=ColumnType.NUMERICAL),
threshold=40.0,
missing_evaluation=False),
pos_child=NonLeafNode(
condition=CategoricalIsInCondition(
feature=SimpleColumnSpec(name="island",type=ColumnType.CATEGORICAL),
mask=["Dream", "Torgersen"],
missing_evaluation=False)
,pos_child=LeafNode(value=ProbabilityValue(probability=[0.8, 0.2], num_examples=10))
,neg_child=LeafNode(value=ProbabilityValue(probability=[0.1, 0.9], num_examples=20))
),
neg_child=LeafNode(value=ProbabilityValue(probability=[0.2, 0.8], num_examples=30))
)
)
)
builder.close()
manual_model = tf.keras.models.load_model(model_path)
# Convert the pandas dataframe into a tf dataset.
dataset_df['species_binary'] = dataset_df['species'] == 'Adelie'
dataset_tf_2 = tfdf.keras.pd_dataframe_to_tf_dataset(dataset_df[['bill_length_mm','island','species_binary']], label="species_binary")
# model compile and fit
manual_model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=1e-3),
loss=tf.keras.losses.BinaryCrossentropy(),
metrics=[tf.keras.metrics.BinaryAccuracy(),
tf.keras.metrics.FalseNegatives()])
manual_model.fit(dataset_tf_2)
Questions
-
The above code runs without error, but the tree does not reflect the fitting results, the prediction prob and number of samples stay the same, which is very weird, looks like the
manual_modelis a completely static model. How can we define the prob and number of samples before running the model on some data? -
I assume that the
tfdf.builder.CARTBuilderis used to build a shell, and the performance of each node can be reflected after fitting/prediction. I am very confused why it requires me to define the value in the leaf in the first place and the value remains the same after fitting/prediction, did I miss anything? -
What is the best practice to use
tfdf.builder.CARTBuilderto build a decision tree by hand on earth?