How are bool tensors stored in tensorflow?

slai-nick · May 16, 2022, 11:58am

How does tensorflow stores/represents bool tensors in memory?

Bhack · May 16, 2022, 12:15pm

What do you meant?

tensorflow/tensorflow/blob/master/tensorflow/core/framework/types.cc

/* Copyright 2015 The TensorFlow Authors. All Rights Reserved.

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
==============================================================================*/

#include "tensorflow/core/framework/types.h"
#include "tensorflow/core/framework/register_types.h"
#include "tensorflow/core/lib/strings/str_util.h"
#include "tensorflow/core/lib/strings/strcat.h"
#include "tensorflow/core/platform/logging.h"

This file has been truncated. show original

slai-nick · May 16, 2022, 1:34pm

I haven’t been very clear effectively, what I meant to ask was what is the size of a bool element in tensorflow?

Bhack · May 16, 2022, 1:56pm

Isn’t just c++ bool?

github.com

tensorflow/tensorflow/blob/master/tensorflow/core/framework/types.cc#L302

      
        
            DEFINE_DATATYPETOENUM_VALUE(uint32);
            DEFINE_DATATYPETOENUM_VALUE(uint16);
            DEFINE_DATATYPETOENUM_VALUE(uint8);
            DEFINE_DATATYPETOENUM_VALUE(int16);
            DEFINE_DATATYPETOENUM_VALUE(int8);
            DEFINE_DATATYPETOENUM_VALUE(tstring);
            DEFINE_DATATYPETOENUM_VALUE(complex64);
            DEFINE_DATATYPETOENUM_VALUE(complex128);
            DEFINE_DATATYPETOENUM_VALUE(int64_t);
            DEFINE_DATATYPETOENUM_VALUE(uint64);
            DEFINE_DATATYPETOENUM_VALUE(bool);
            DEFINE_DATATYPETOENUM_VALUE(qint8);
            DEFINE_DATATYPETOENUM_VALUE(quint8);
            DEFINE_DATATYPETOENUM_VALUE(qint16);
            DEFINE_DATATYPETOENUM_VALUE(quint16);
            DEFINE_DATATYPETOENUM_VALUE(qint32);
            DEFINE_DATATYPETOENUM_VALUE(bfloat16);
            DEFINE_DATATYPETOENUM_VALUE(Eigen::half);
            DEFINE_DATATYPETOENUM_VALUE(ResourceHandle);
            DEFINE_DATATYPETOENUM_VALUE(Variant);
            #undef DEFINE_DATATYPETOENUM_VALUE

slai-nick · May 16, 2022, 2:31pm

Ok thank you.
I just wanted to check that there wasn’t any optimisation trick like in C++ std::vector<bool> (which can use bit masking in some implementations).

I have a little concern about this though, it seems that sizeof(bool) isn’t a C++ language standard.

Bhack · May 16, 2022, 2:46pm

I think you could check:

github.com

tensorflow/tensorflow/blob/master/tensorflow/core/framework/tensor.cc#L18-L24

      
        
            // Tensor.cc uses a few templated classes and structs to facilitate
            // implementation of the Tensor class.
            //
            // * Buffer<T>: provides the implementation for a typed array T[n].
            //   The array is allocated by the given allocator. It runs T's
            //   default constructors and destructors when T is not a simple type
            //   (e.g., string.), and skips them otherwise.

slai-nick · May 16, 2022, 3:01pm

Thank you for your help!