Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

StackOverflowError when many items to fetch? #149

Closed
ALPSMAC opened this issue Mar 14, 2018 · 3 comments
Closed

StackOverflowError when many items to fetch? #149

ALPSMAC opened this issue Mar 14, 2018 · 3 comments

Comments

@ALPSMAC
Copy link

ALPSMAC commented Mar 14, 2018

Exception in thread "ForkJoinPool-1-worker-4" java.lang.StackOverflowError
	at scala.runtime.BoxesRunTime.equals2(BoxesRunTime.java:135)
	at scala.runtime.BoxesRunTime.equals(BoxesRunTime.java:121)
	at scala.collection.immutable.HashMap$HashMap1.get0(HashMap.scala:200)
	at scala.collection.immutable.HashMap$HashTrieMap.get0(HashMap.scala:327)
	at scala.collection.immutable.HashMap$HashTrieMap.get0(HashMap.scala:327)
	at scala.collection.immutable.HashMap$HashTrieMap.get0(HashMap.scala:322)
	at scala.collection.immutable.HashMap.get(HashMap.scala:53)
	at fetch.InMemoryCache.get(cache.scala:43)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:90)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:87)
	at cats.free.FreeTopExt$.modify(freeinspect.scala:51)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:95)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:87)
	at cats.free.FreeTopExt$.modify(freeinspect.scala:51)
	at cats.free.FreeTopExt$.modify(freeinspect.scala:52)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:96)
	at fetch.interpreters.ParallelJoinPhase$$anon$2.apply(ParallelJoinPhase.scala:87)
	at cats.free.FreeTopExt$.modify(freeinspect.scala:51)
	at cats.free.FreeTopExt$.modify(freeinspect.scala:52)
        ...

Cache implementation is thus:

  case object PassthroughCache extends fetch.DataSourceCache {
    override def update[A](k: fetch.DataSourceIdentity, v: A): DataSourceCache = this
    override def get[A](k: fetch.DataSourceIdentity): Option[A] = None
  }

Not quite sure what exactly is causing the Stack Overflow here, or if we can do anything about it... Any ideas? Happy to help address the issue if someone can point me in the right direction.

@ALPSMAC
Copy link
Author

ALPSMAC commented Mar 15, 2018

Thinking that perhaps the cause was due to interpreting to a Future (which I know has stack safety issues), I tried converting over to interpreting into a Monix Task. No luck though. Still getting the same Stack Overflow error.

@ALPSMAC
Copy link
Author

ALPSMAC commented Mar 15, 2018

Additional Info: I am performing a fetch of 3600 items via a fetchMany operation, batched into groups of 360, with parallel batch execution.

@ALPSMAC
Copy link
Author

ALPSMAC commented Mar 15, 2018

Okay - I've done some more poking and if I up the JVM stack size I can get the test case to pass. Closing as a limitation, but not an issue.

@ALPSMAC ALPSMAC closed this as completed Mar 15, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant