Support writes if dimensions are different but data size is compatible

[emmenlau]

I would like to kindly request a feature. Currently I can not write i.e. a 1D buffer of size [6] into a 2D dataset of dimensions [6, 1]. I see how the current behavior makes sense from a point of view. But the number of elements and their type would be compatible. Therefore it would be great to transparently support this.

My suggestion: Support all writes, if at least one of the following is true:

• The dimensions match (current behaviour).
• The number of elements match, and the mismatching dimensions have size 1 (new feature request).

I understand that if at least one of the two is true, then the arrays can be considered "compatible". This would allow to treat i.e. a 2D vector with a single row or column the same as a 1D vector. This is something my users commonly do. It would be great to support this transparently.

What do you think?

[1uc]

We used to have the feature in v2. It used to do numpy-style broad-casting, but from the wrong end (because that's the useful choice in this context). For example the version you're proposing I think doesn't work.

Let's say you have an array for p(t_k, x_i, y_j) == p[k, i, j]. Since you know that the in-memory buffer is one time-slice you store it as [n, m]. It all works pretty nicely, until one day, unfortunately, m == 1. HighFive is typically able to reshape/reallocate arrays, therefore when it's faces with a three-dimensional array with shape [1, n, 1] it has to decide: does it read it into [n, 1] or into [1, n]. Since Highfive can't know that the array is a time-slice, it can't know that [n, 1] is correct and [1, n] is wrong.

In HighFive, there's two functions you can use: squeezing and reshaping.

See: https://highfive-devs.github.io/highfive/md__2home_2runner_2work_2highfive_2highfive_2doc_2migration__guide.html

If you're able to find a way of dealing with this that's intuitive, maintainable and not error prone, I'm willing to consider it. Personally, I've given up on the idea for HighFive. The one exception might be into 1D data.