ByRow without columns

[matthieugomez]

Would it be possible at some point to use DataFrames ByRow without columns? This would allow one to do things like:

transform(df, Any[] => ByRow(rand) => :z)

[bkamins]

It was considered.

The reason why it is not allowed is that ByRow is essentially a shorthand for broadcasting currently. To keep consistency here broadcasting over an empty collection would give us an empty collection. If you pass at least one column then you are good as this column defines you the dimensions of the output.

Actually your example was the only use-case I could come-up with that would be useful and I thought that it is not worth to break the design to handle it.

Technically it would be possible to work around this and add this support but I was not sure if it is worth it. What do you think?

[pdeffebach]

I would like it. I was working on a problem set the other day where I wanted to generate random numbers for error terms.

This actually seems quite hard to fix! we have to catch the combination of Any[] and ByRow and then use some sort sort of struct for dispatch that constains the number of rows in a DataFrame to call [f() for i in 1:N]

[pdeffebach]

It will also make DataFramesMeta a lot easier in case you see something with the yet-to-be-implemented @row macro

@transform(df, @row y = 7)

This will get lowered to

transform(df, [] => ByRow(_f) => :y)

which will error.

[bkamins]

This actually seems quite hard to fix!

If we add it (I will add it then) we have to hard-code this in every function that potentially might call ByRow.

Note that ByRow(sin) is just a function. It knows nothing about data frames existence at all and if x is a vector ByRow(sin)(x) is just the same as writing sin.(x) (and has nothing to do with data frames).

[matthieugomez]

Yes, exactly. Ideally macro-packages will make it as easy to work row-wise and col-wise (that's the point of PairsMacros at least), and so people do end up writing `@rows y = 1`. I don't think that needs to be done before 1.0, but it's certainly something that would be nice to have someday.

…

On Tue, Sep 29, 2020 at 6:39 AM pdeffebach ***@***.***> wrote: It will also make DataFramesMeta a lot easier in case you see something with the yet-to-be-implemented @Row macro @Transform(df, @Row y = 7) This will get lowered to transform(df, [] => ByRow(_f) => :y) which will error. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#2457 (comment)>, or unsubscribe </~https://github.com/notifications/unsubscribe-auth/ABPPPXO5VOKSGWVYMP5Y2Z3SIHPQLANCNFSM4R5AKFRQ> .

[bkamins]

I have added it to the PR fixing #2410.
In a few days I will open an issue for #2410 for AbstractDataFrame. Then looking at it will be welcome.
When we finalize that one I will open another PR for GroupedDataFrame (where we also need to change this in particular) and filter.