Can I give dictionaries or tuples as observation_spec?

Research & Models
,
#1

I want to use REINFORCE and I’m creating an enviroment. I want to train the model to get to the top of a mountain, and it has 4 hikers that it can move to explore the map (it doesn’t know the mountain). I can define observation_specs like this?

self._observation_spec = array_spec.BoundedArraySpec(shape=(4,), dtype=dict, name='observation')

The idea is that the thing that outputs looks like this:
{“hiker1”: (x,y,z,dx,dy), “hiker2”: (x,y,z,dx,dy), “hiker3”: (x,y,z,dx,dy), “hiker4”: (x,y,z,dx,dy)}

It could also be given like this?

self._observation_spec = array_spec.BoundedArraySpec(shape=(4,), dtype=np.array, name='observation')

np.array[(x,y,z,dx,dy),(x,y,z,dx,dy),(x,y,z,dx,dy),(x,y,z,dx,dy)]

#3

Hi @Mankitto ,

While I am not an expert in the Reinforce algorithm, I did some research within the available resources, and here are my learnings:

Yes, both of those are valid ways to define the observation_spec for your environment. The first one uses a dictionary to store the information about each hiker, and the second one uses an array to store the information about all four hikers.

The first one is probably the easiest to understand, but the second one is more efficient because it doesn’t require any extra data structures.

If you come across any additional information or have further questions, feel free to share.

I hope this helps!

Thanks