Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reduce stack consumption #95

Merged
12 commits merged into from
Mar 24, 2017
Merged

Reduce stack consumption #95

12 commits merged into from
Mar 24, 2017

Conversation

ghost
Copy link

@ghost ghost commented Dec 2, 2016

  • Query aggregation
  • Join node simplification

Tried some extreme cases and found a couple of parts in the code where we could reduce stack consumption. Unfortunately the tests are failing due to a timeout in Travis CI, looking into it right now.

@ghost ghost force-pushed the stack-safety branch 6 times, most recently from 3e321a0 to 92d9c7e Compare January 5, 2017 11:16
@ghost ghost force-pushed the stack-safety branch from fff5ffb to 1c023ad Compare March 23, 2017 12:25
@codecov-io
Copy link

codecov-io commented Mar 23, 2017

Codecov Report

Merging #95 into master will increase coverage by 0.49%.
The diff coverage is 88.88%.

@@            Coverage Diff             @@
##           master      #95      +/-   ##
==========================================
+ Coverage    77.1%   77.59%   +0.49%     
==========================================
  Files          10       10              
  Lines         249      250       +1     
  Branches        2        2              
==========================================
+ Hits          192      194       +2     
+ Misses         57       56       -1
Impacted Files Coverage Δ
shared/src/main/scala/interpreters.scala 93.93% <88.88%> (+0.8%) ⬆️

Continue to review full report at Codecov.

Legend - Click here to learn more
Δ = absolute <relative> (impact), ø = not affected, ? = missing data
Powered by Codecov. Last update 61bed1d...720abcc. Read the comment docs.

@ghost ghost requested review from raulraja and peterneyens March 23, 2017 14:10
Copy link
Contributor

@raulraja raulraja left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💚

@raulraja
Copy link
Contributor

This looks great, looks like codecov says it decreased in 3% though and bumps it below 80% which we may want to address if it's something we can do as part of this PR. This improvements deserves a new release 👏

Copy link
Contributor

@peterneyens peterneyens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The stack safety changes look great 👍 , but it seems that we have lost a few of our tests while splitting them in multiple files?

.map((x) => (0 until 2).toList.traverse(fetchBatchedDataSeq))
.foldLeft(
Fetch.pure(List.empty[Int])
)((acc, f) => acc.flatMap((x) => f))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This foldLeft with pure and flatMap could also be written using the monadic fold (Foldable#foldM).

Something like (might need some type annotations):

import cats.instances.list._
import cats.syntax.foldable._

.foldM(List.empty[Int])((_, f) => f)
// or using foldMapM (this does combine all the lists though)
.foldMapM(identity)

}
}

class FetchTests extends AsyncFreeSpec with Matchers {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have we dropped these ?
Seems we are removing around 550 lines in this PR, which is more or less the same as these tests ?

@ghost
Copy link
Author

ghost commented Mar 24, 2017

@peterneyens i added the lost tests back and changed the foldLeft usage for foldM, check it out please!

@ghost ghost merged commit 5e26f7c into master Mar 24, 2017
@ghost ghost deleted the stack-safety branch March 24, 2017 11:57
@ghost
Copy link
Author

ghost commented Mar 24, 2017

:shipit:

This pull request was closed.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants