i am trying to load my datasets using a csv file using the DataLoader.from_csv('path to csv', images_dir = 'path to image directory) however when i go on to start training training the model i get an error AttributeError: 'NoneType' object has no attribute 'label_map'. it seems like whatever from returned from Dataloader.from_csv` was a None type
Hi @Ebyau_Bryan ! Can you show the code you are using to load the data. The function returns None only if it doesn’t find correct train, training,test data in the CSV.
This is my code for loading data from csv
from tflite_model_maker.object_detector import DataLoader
train_dir = '/content/drive/MyDrive/Training_demo/images/my_dataset/train'
train_data, validation_data, test_data = DataLoader.from_csv('/content/drive/MyDrive/Training_demo/annotations/train_dataset.csv', images_dir = train_dir)@Ebyau_Bryan there should be no error if train_dir and train_dataset.csv exist. could you try loading the csv to pandas dataframe.
import pandas as pd
mycsv = pd.read_csv ('path')
mycsv.head()
and check if csv has loaded correctly
the csv file loads successfully,
it would be nice if I could see your collab file. Sorry, I should have asked about this earlier it would have been easier and faster for you.
@Ebyau_Bryan It works just fine for me on my dummy dataset… I don’t have your csv file or data but it seems like it is not in the format DataLoader is expecting pls look into Formatting a training data CSV | AutoML Vision Object Detection (google.com)
Alright . Are there scripts to format data into the required format for DataLoader otherthan using Google cloud AutoML Object Detection
Hi.
I had the same error and managed to identify that it was due to several factors:
- The file generated by the tagging tool did not have the required format, which is why I had to convert the file to the format specified in https://cloud.google.com/vision/automl/object-detection/docs/csv- format? hl = cs. To do this, use (provisionally) a spreadsheet (TensorFlow Lite - Data Preparation Template - Google Sheets).
- The value UNASSIGNED cannot be used. Each row must be specified as TRAINING, VALIDATION, or TEST.
Converting with a spreadsheet creates other minor issues that I fixed with a plain text editor. In the next few days I will create a Python script to convert to the indicated format.
Just wanted to say that I had the same issue and also fixed it by manually allocating my labels to TRAIN, VALIDATION or TEST, rather than relying on UNASSIGNED to do it for me.
Not sure if I’m misunderstanding the documentation—perhaps TFLite’s CSV loader is linking to the AutoML documentation because it uses the same spec but just doesn’t support the UNASSIGNED feature? The docs do say. ‘assigned to one of the above three sets by AutoML Vision Object Detection’ (emphasis mine). If this is indeed the case, perhaps the link from TFLite should note this qualification.
Is this CSV format any type of standard or has any name that could be referred to?
Also the import function over at AutoML console did not really offer much help.
This is the format specified for TFLite specified at Formatting a training data CSV | AutoML Vision Object Detection | Google Cloud.
As I am not using AutoML, I had to modify the file manually.
I met the same issue, my original csv file was generated by Label Studio, and it even can’t be loaded by pd.read_csv directly, after I loaded this csv file ,and did some operations, I used pd.DataFrame.to_csv to saved it to another csv file, the content of one line in this new csv file is :
6,2023-05-19T06:31:46.991379Z,773.156,“[{‘x’: 25.370923632275424, ‘y’: 27.107188552444285, ‘width’: 10.83937007112304, ‘height’: 5.504587155963301, ‘rotation’: 352.793942998617, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 37.34770293330205, ‘y’: 25.543665713358397, ‘width’: 11.284403669724759, ‘height’: 8.440366972477081, ‘rotation’: 353.354721789727, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 49.53425282025861, ‘y’: 24.23680278573774, ‘width’: 11.763393576347777, ‘height’: 8.639269419131317, ‘rotation’: 350.18773196696736, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 26.375280367933488, ‘y’: 39.57903041235135, ‘width’: 10.746377295037547, ‘height’: 6.28877502893554, ‘rotation’: 352.2927950502482, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 38.425220449831244, ‘y’: 35.23646392313965, ‘width’: 11.255866157151425, ‘height’: 9.36417294391476, ‘rotation’: 350.7637916142988, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 51.426826019202, ‘y’: 33.14280277307643, ‘width’: 10.484930491137563, ‘height’: 8.70546432561436, ‘rotation’: 352.26438093510166, ‘rectanglelabels’: [‘Object_y’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 26.339497849873766, ‘y’: 47.277900710040875, ‘width’: 11.37945347399733, ‘height’: 8.440006993587096, ‘rotation’: 351.1095767426363, ‘rectanglelabels’: [‘Object_o’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 39.234767629612165, ‘y’: 45.25084430318517, ‘width’: 11.734301439370988, ‘height’: 8.192241808139237, ‘rotation’: 351.1095767426374, ‘rectanglelabels’: [‘Object_o’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 52.221570519171145, ‘y’: 42.902529968269214, ‘width’: 11.188693862668936, ‘height’: 9.028805560365935, ‘rotation’: 349.18612248637714, ‘rectanglelabels’: [‘Object_o’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 22.962670070953145, ‘y’: 27.32295875773547, ‘width’: 38.68517845705243, ‘height’: 29.24360234164902, ‘rotation’: 351.33999310253387, ‘rectanglelabels’: [‘Object_sd’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 28.196107645674857, ‘y’: 58.077194965486285, ‘width’: 11.108449990715524, ‘height’: 6.505015955036575, ‘rotation’: 349.69515353123506, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 41.18583628132604, ‘y’: 55.51254148379392, ‘width’: 11.336380521873513, ‘height’: 7.0111876281364856, ‘rotation’: 351.57303097851957, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 54.91257525655315, ‘y’: 53.67035969286604, ‘width’: 10.183486238532094, ‘height’: 6.457412499831959, ‘rotation’: 350.50942169550956, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 42.808980104091965, ‘y’: 63.23830650785564, ‘width’: 11.377856030968083, ‘height’: 6.686068718899433, ‘rotation’: 351.7732775358926, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 56.394715484011286, ‘y’: 61.07380641047717, ‘width’: 10.286295008179376, ‘height’: 6.425319140785042, ‘rotation’: 351.31364541876496, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 45.191985779458, ‘y’: 71.07348241305323, ‘width’: 10.917431192660528, ‘height’: 5.641613452410933, ‘rotation’: 350.7370400417704, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 58.06752019685683, ‘y’: 68.32478804520957, ‘width’: 9.203853577802978, ‘height’: 6.216555249102603, ‘rotation’: 351.1126867246379, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 61.28440366972477, ‘y’: 59.3577981651376, ‘width’: 0.6422018348623854, ‘height’: 0.3669724770642202, ‘rotation’: 0, ‘rectanglelabels’: [‘Object_b’], ‘original_width’: 224, ‘original_height’: 224}, {‘x’: 27.630966171290506, ‘y’: 56.8147488509194, ‘width’: 38.71906644490655, ‘height’: 23.164341657817673, ‘rotation’: 351.10414303034, ‘rectanglelabels’: [‘Object_sd’], ‘original_width’: 224, ‘original_height’: 224}]”,2023-05-19T06:31:46.991403Z,2,66c4b7b1-CFrames100001.jpg,1,TRAIN
Then I used object_detector.DataLoader.from_csv to create the dataloaders , the dataloaders are all NONE,
By the way, I was hoping to detect multiple objects, so there might be more than one box in this row of content. It is the reason makes this issue happen?