Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Window, WindowLeft and WindowRight are not optimal in memory #627

Closed
Orace opened this issue Oct 31, 2019 · 7 comments
Closed

Window, WindowLeft and WindowRight are not optimal in memory #627

Orace opened this issue Oct 31, 2019 · 7 comments

Comments

@Orace
Copy link
Contributor

Orace commented Oct 31, 2019

Window, WindowLeft and WindowRight take O(sourceSize*windowSize) in memory.
If we accept that the method returns IEnumerable<IReadOnlyList<TSource>> instead of IEnumerable<IList<TSource>>, we can go down to O(consumedElement) < O(sourceSize).

Orace added a commit to Orace/MoreLINQ that referenced this issue Oct 31, 2019
@Orace Orace changed the title WindowLeft and WindowRight made to many memory copy Window, WindowLeft and WindowRight made to many memory copy Oct 31, 2019
@Orace Orace changed the title Window, WindowLeft and WindowRight made to many memory copy Window, WindowLeft and WindowRight are not optimal in memory Oct 31, 2019
@Orace
Copy link
Contributor Author

Orace commented Oct 31, 2019

@atifaziz is there any equivalent to WindowLeftAndRight available ?

@atifaziz
Copy link
Member

atifaziz commented Nov 1, 2019

is there any equivalent to WindowLeftAndRight available ?

No, never needed it and no one ever asked for it. Do you have a use case?

@Orace
Copy link
Contributor Author

Orace commented Nov 1, 2019

I was looking for it to improve it ;)
I think a Window(int size, bool extendLeft = false, bool extendRight = false, bool extendBoth = false) overload is missing here.

The behavior will be

extendLeft |= extendBoth;
extendRight |= extendBoth;

The usages pretty straightforward.

@atifaziz
Copy link
Member

atifaziz commented Nov 1, 2019

For dynamic behaviour? I think if someone needs that, they can achieve it like this:

var xs = Enumerable.Range(1, 5);
var windows =
    from f in new Func<IEnumerable<int>, int, IEnumerable<IList<int>>>[]
    {
        MoreEnumerable.Window,
        MoreEnumerable.WindowLeft,
        MoreEnumerable.WindowRight,
    }
    select f(xs, 3).Select(w => $"[{w.ToDelimitedString(", ")}]")
                   .ToDelimitedString(", ");
foreach (var ws in windows)
    Console.WriteLine(ws);
// output:
// [1, 2, 3], [2, 3, 4], [3, 4, 5]
// [1, 2, 3], [2, 3, 4], [3, 4, 5], [4, 5], [5]
// [1], [1, 2], [1, 2, 3], [2, 3, 4], [3, 4, 5]

@Orace
Copy link
Contributor Author

Orace commented Nov 1, 2019

My proposal is WindowLeft(size) as an alias for Window(size, extendLeft: true), same thing for right. And a solution for WindowLeftAndRight (should we create the alias is an open question). All of this can fit in a single implementation (less code to maintain).

It facilitate the cases where the choice is made at runtime: Window(size, extendRight: isTailNeed).

We have to discuss the name of the optional extendLeft, extendRight, extendBoth parameters.

@Orace
Copy link
Contributor Author

Orace commented Nov 2, 2019

Segment also performs data copy. I seen an issue about it but I can't find it anymore.

@Orace
Copy link
Contributor Author

Orace commented Nov 6, 2019

Window return a sequence of R/W enable fresh new array and there is no easy way to maintain a Read only version with memory optimisation (keep track of disposed window to free memory in the back end buffer give segmentation management problems).
With the current implementation the GC do all the job. It is O((seqLength-windowSize)*windowSize) so it is O(seqLength²) in worst case.

We might need BigO information in the documentation.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants