Running TFLite micro on BE platform

Special Interest Groups Micro
,
#1

Hello,
I am trying to use TFLM on a platform using big endian. The tensor allocation fails as according to commit #689 the support for BE was removed. A custom tool for the endianness conversion of the .tflite file is proposed to use as solution. Since the model in .tflite format is a flatbuffer, I rebuilt schema_generated.h using the tflite schema with the mutable option on. I then tried to use the non-const accessors to load the data and byte-swap it, before saving it again. However I can’t seem to make it work.

I was trying something like this:

  //model_as_char_array obtained using the xxd command
  model = tflite::GetMutableModel(model_as_char_array); 
  auto vec=model->mutable_buffers();
  for (auto it = vec->begin(); it != vec->end(); ++it){
    auto d=it->mutable_data();// <----the problem is on this line
}

which fails to compile due to invalid conversion (const to non-const):

error: invalid conversion from 'flatbuffers::IndirectHelper<flatbuffers::Offset<tflite::Buffer> >::return_type' {aka 'const tflite::Buffer*'} to 'tflite::Buffer*' [-fpermissive]
   74 |   IT operator->() const { return IndirectHelper<T>::Read(data_, 0); }
      |                                  ~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~
      |                                                         |
      |                                                         flatbuffers::IndirectHelper<flatbuffers::Offset<tflite::Buffer> >::return_type {aka const tflite::Buffer*}

I am obviously missing something and got stuck on this hard. Can anyone propose a way to achieve the aforementioned conversion? Any help is highlt appreciated.
Thank you!

Note: While flattbuffers perform the byte-swap when the accessors are used, TFLM (probably) tries to read the buffer directly and hence the byte-swap is not performed.

#3

After some further investigation, I managed to resolve the problem. Posting the solution if anyone else runs into similar problems. The steps assume only INT32 (or other 4-bytes data types such as FLOAT)
are used for the model parameters.

  1. swap the bytes inside the buffer using the mutable accessors or C API:
  tflite::ModelT model;
  tflite::GetModel(MODEL)->UnPackTo(&model);
  uint32_t sizeBuffer = model.buffers.size();

  for (uint32_t i = 0; i < sizeBuffer; i++)
  {
    uint32_t sizeData = model.buffers.at(i)->data.size();
    for (uint32_t j = 0; j < sizeData; j += 4)
    {
      uint8_t vals[] = {
          model.buffers.at(i)->data[j + 0],
          model.buffers.at(i)->data[j + 1],
          model.buffers.at(i)->data[j + 2],
          model.buffers.at(i)->data[j + 3],
      };

      model.buffers.at(i)->data[j + 0] = vals[3];
      model.buffers.at(i)->data[j + 1] = vals[2];
      model.buffers.at(i)->data[j + 2] = vals[1];
      model.buffers.at(i)->data[j + 3] = vals[0];
    }
  }
  flatbuffers::FlatBufferBuilder builder;
  builder.Finish(tflite::Model::Pack(builder, &model));
  uint8_t *buf = builder.GetBufferPointer();
  int size = builder.GetSize();

  std::ofstream ofile(name, std::ios::binary);
  ofile.write((char *)buf, size);
  ofile.close();
  1. Edit the flatbuffer_utils.cc file by modifying FlatBufferVectorToTfLiteTypeArray methods (similarly for the float array):
/// container to hold the allocated TfLiteInt arrays to avoid memory leak
std::map<const flatbuffers::Vector<int32_t>*, TfLiteIntArray*>
    TfliteIntArrayMap;

TfLiteIntArray* FlatBufferVectorToTfLiteTypeArray(
    const flatbuffers::Vector<int32_t>* flatbuffer_array) {
  // if the key doesnt exist
  if (TfliteIntArrayMap.count(flatbuffer_array) == 0) {
    int size = flatbuffer_array->size();
    // allocate memory for it
    size_t alloc_size = TfLiteIntArrayGetSizeInBytes(size);
    if (alloc_size <= 0) return nullptr;
    TfLiteIntArray* ret = (TfLiteIntArray*)malloc(alloc_size);
    if (!ret) return ret;
    ret->size = size;
    for (int i = 0; i < size; i++) {
      ret->data[i] = flatbuffer_array->Get(i);
    }
    // save it to map
    TfliteIntArrayMap.insert(
        std::pair<const flatbuffers::Vector<int32_t>*, TfLiteIntArray*>(
            flatbuffer_array, ret));
  }
  // return the map element for the given flatbuffer::Vector
  return TfliteIntArrayMap[flatbuffer_array];
}

Note that byteswapping the subgraphs as well won’t work, as TFLM loads vectors from those buffers as well as reinterpret_cast it to TfLiteArray, so in one of those cases, the resulting container will have its size swapped anyway.

It’s not the cleanest solution, but seems to give correct results.