innerjoin fast path where join column is allequal?

[anandijain]

ts_ = map(x->x.t, dfs)
@test allequal(ts_)
@test all(issorted, ts_)
@test length(dfs) == 11128
@test size(dfs[1]) == (5761, 6)
j = innerjoin(dfs...;on=:t)

Currently I get a stackoverflow and crash when attempting a join of this size (~2.5GB). What is the recommended way to do big joins of this nature?

I don't have a MWE, but it should be relatively easy to make the repro for this.

[bkamins]

StackOverflow is most likely because innerjoin uses recursion in this case and is unrelated with data size.
Instead of splatting do the join in iteration:

foldl((df1, df2) -> innerjoin(df1, df2, on=:t), dfs)

However, if allequal(ts_) is true then instead you can just do insertcols! as it should be faster (or hcat).

[anandijain]

Is there a reason to not define innerjoin(dfs::Vector) and just have that call fold?

[bkamins]

I will make a fix to this (the reason why we used recursion is that we did not assume such wide inputs)