Good day all.
I have been following along with the TextSearcher tutorial on Google Colab. Despite some hiccups in running the tflite_model_maker in Colab, I was able to find a workaround that allows the successful completion.
The next step, where I am having issues, is the use of a custom dataset in lieu of the content in the tutorial.
When trying to load my custom CSV, I keep getting a rather obtuse error:
'NoneType' object has no attribute 'shape'
I am following along with a script that contains all of the instructions from the tutorial but I am unable to get to the root of the issue.
# This script is used to train the model using the TFLite Model Maker library.
# The intention is to use this script on the SupportContent Google Colab notebook.
# DO NOT USE THIS LOCALLY
from tflite_model_maker import searcher
from tflite_support.task import text
import os
SUPPORT_CONTENT_TFLITE_FILE = "support_content.tflite"
SUPPORT_CONTENT_CSV_FILE = "support_content.csv"
def pretty_print(statement):
print("*******************************************", end='\n\n')
print(statement, end='\n\n')
print("*******************************************", end='\n\n')
try:
pretty_print("Start training...")
data_loader = searcher.TextDataLoader.create(
"universal_sentence_encoder.tflite",
l2_normalize=True)
if data_loader is None:
raise Exception("Failed to create TextDataLoader object")
data_loader.load_from_csv(
SUPPORT_CONTENT_CSV_FILE,
text_column="content",
metadata_column="url")
if data_loader.__len__ == 0:
raise Exception("data_loader.dataset is empty")
pretty_print("Configure ScaNN")
scann_options = searcher.ScaNNOptions(
distance_measure="dot_product",
score_brute_force = searcher.ScoreBruteForce())
if scann_options is None:
raise Exception("Failed to configure ScaNN and create ScaNNOptions object")
print("scann_options:", scann_options)
pretty_print("Create model")
support_content_searcher = searcher.Searcher.create_from_data(
data_loader,
scann_options)
if support_content_searcher is None:
raise Exception("Failed to create Searcher instance from data_loader and scann_options")
print("model:", support_content_searcher)
pretty_print("Export model")
tflite_export_filename = f"{SUPPORT_CONTENT_TFLITE_FILE}"
print("Exporting model to", tflite_export_filename)
support_content_searcher.export(
export_format=searcher.ExportFormat.TFLITE,
export_filename=tflite_export_filename,
userinfo="")
if os.path.exists(tflite_export_filename):
pretty_print(f"{tflite_export_filename} file was created")
else:
raise Exception(f"{tflite_export_filename} file was not created")
pretty_print("Testing model...")
# Initializes a TextSearcher object.
searcher = text.TextSearcher.create_from_file(tflite_export_filename)
# Searches the input query.
results = searcher.search("I want to close my account.")
pretty_print(results)
except ValueError as ve:
pretty_print(f"A ValueError was raised: {ve}")
except Exception as e:
pretty_print(f"An exception was raised: {e}")
The script works without issue until it gets to the export function where an exception is raised and caught. This is where the confusing, error message is retrieved from.
Is there any way I can try and delve deeper into what is going on here to try and get to the root cause.
I have tried reviewing the source of tflite_model_maker (Github repo here) but there is nothing obvious.
I am hoping somebody on here can point me in the right direction.
Thanks