Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Flaky test TestCopyOnWriteMerge > testMergeWithSnapshotIsolation() #11066

Closed
1 of 3 tasks
manuzhang opened this issue Sep 2, 2024 · 0 comments
Closed
1 of 3 tasks

Flaky test TestCopyOnWriteMerge > testMergeWithSnapshotIsolation() #11066

manuzhang opened this issue Sep 2, 2024 · 0 comments
Labels
bug Something isn't working

Comments

@manuzhang
Copy link
Collaborator

Apache Iceberg version

main (development)

Query engine

Spark

Please describe the bug 🐞

16:30:41 TestCopyOnWriteMerge > testMergeWithSnapshotIsolation() > catalogName = testhive, implementation = org.apache.iceberg.spark.SparkCatalog, config = {type=hive, default-namespace=default}, format = ORC, vectorized = true, distributionMode = none, fanout = true, branch = main, planningMode = LOCAL FAILED
16:30:41     java.util.concurrent.ExecutionException: org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.iceberg.spark.extensions.TestMerge that uses java.util.concurrent.atomic.AtomicInteger, java.util.concurrent.atomic.AtomicIntegerint was not fulfilled within 5 seconds.
16:30:41         at java.util.concurrent.FutureTask.report(FutureTask.java:122)
16:30:41         at java.util.concurrent.FutureTask.get(FutureTask.java:192)
16:30:41         at org.apache.iceberg.spark.extensions.TestMerge.testMergeWithSnapshotIsolation(TestMerge.java:1684)
16:30:41         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16:30:41         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16:30:41         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16:30:41         at java.lang.reflect.Method.invoke(Method.java:498)
16:30:41         at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:728)
16:30:41         at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
16:30:41         at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
16:30:41         at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:156)
16:30:41         at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMethod(TimeoutExtension.java:147)
16:30:41         at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestTemplateMethod(TimeoutExtension.java:94)
16:30:41         at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker$ReflectiveInterceptorCall.lambda$ofVoidMethod$0(InterceptingExecutableInvoker.java:103)
16:30:41         at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.lambda$invoke$0(InterceptingExecutableInvoker.java:93)
16:30:41         at org.junit.jupiter.engine.execution.InvocationInterceptorChain$InterceptedInvocation.proceed(InvocationInterceptorChain.java:106)
16:30:41         at org.junit.jupiter.engine.execution.InvocationInterceptorChain.proceed(InvocationInterceptorChain.java:64)
16:30:41         at org.junit.jupiter.engine.execution.InvocationInterceptorChain.chainAndInvoke(InvocationInterceptorChain.java:45)
16:30:41         at org.junit.jupiter.engine.execution.InvocationInterceptorChain.invoke(InvocationInterceptorChain.java:37)
16:30:41         at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:92)
16:30:41         at org.junit.jupiter.engine.execution.InterceptingExecutableInvoker.invoke(InterceptingExecutableInvoker.java:86)
16:30:41         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.lambda$invokeTestMethod$7(TestMethodTestDescriptor.java:218)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.invokeTestMethod(TestMethodTestDescriptor.java:214)
16:30:41         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:139)
16:30:41         at org.junit.jupiter.engine.descriptor.TestMethodTestDescriptor.execute(TestMethodTestDescriptor.java:69)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
16:30:41         at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
16:30:41         at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:226)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask$DefaultDynamicTestExecutor.execute(NodeTestTask.java:204)
16:30:41         at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:142)
16:30:41         at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.lambda$execute$2(TestTemplateTestDescriptor.java:110)
16:30:41         at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
16:30:41         at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
16:30:41         at java.util.stream.ReferencePipeline$2$1.accept(ReferencePipeline.java:175)
16:30:41         at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
16:30:41         at java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
16:30:41         at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
16:30:41         at java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:948)
16:30:41         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
16:30:41         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
16:30:41         at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
16:30:41         at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
16:30:41         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
16:30:41         at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
16:30:41         at java.util.stream.ReferencePipeline$7$1.accept(ReferencePipeline.java:270)
16:30:41         at java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
16:30:41         at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
16:30:41         at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
16:30:41         at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
16:30:41         at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
16:30:41         at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
16:30:41         at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
16:30:41         at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:110)
16:30:41         at org.junit.jupiter.engine.descriptor.TestTemplateTestDescriptor.execute(TestTemplateTestDescriptor.java:44)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:151)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
16:30:41         at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
16:30:41         at java.util.ArrayList.forEach(ArrayList.java:1257)
16:30:41         at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
16:30:41         at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
16:30:41         at java.util.ArrayList.forEach(ArrayList.java:1257)
16:30:41         at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.invokeAll(SameThreadHierarchicalTestExecutorService.java:41)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$6(NodeTestTask.java:155)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$8(NodeTestTask.java:141)
16:30:41         at org.junit.platform.engine.support.hierarchical.Node.around(Node.java:137)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.lambda$executeRecursively$9(NodeTestTask.java:139)
16:30:41         at org.junit.platform.engine.support.hierarchical.ThrowableCollector.execute(ThrowableCollector.java:73)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.executeRecursively(NodeTestTask.java:138)
16:30:41         at org.junit.platform.engine.support.hierarchical.NodeTestTask.execute(NodeTestTask.java:95)
16:30:41         at org.junit.platform.engine.support.hierarchical.SameThreadHierarchicalTestExecutorService.submit(SameThreadHierarchicalTestExecutorService.java:35)
16:30:41         at org.junit.platform.engine.support.hierarchical.HierarchicalTestExecutor.execute(HierarchicalTestExecutor.java:57)
16:30:41         at org.junit.platform.engine.support.hierarchical.HierarchicalTestEngine.execute(HierarchicalTestEngine.java:54)
16:30:41         at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:107)
16:30:41         at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:88)
16:30:41         at org.junit.platform.launcher.core.EngineExecutionOrchestrator.lambda$execute$0(EngineExecutionOrchestrator.java:54)
16:30:41         at org.junit.platform.launcher.core.EngineExecutionOrchestrator.withInterceptedStreams(EngineExecutionOrchestrator.java:67)
16:30:41         at org.junit.platform.launcher.core.EngineExecutionOrchestrator.execute(EngineExecutionOrchestrator.java:52)
16:30:41         at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:114)
16:30:41         at org.junit.platform.launcher.core.DefaultLauncher.execute(DefaultLauncher.java:86)
16:30:41         at org.junit.platform.launcher.core.DefaultLauncherSession$DelegatingLauncher.execute(DefaultLauncherSession.java:86)
16:30:41         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.processAllTestClasses(JUnitPlatformTestClassProcessor.java:110)
16:30:41         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor$CollectAllTestClassesExecutor.access$000(JUnitPlatformTestClassProcessor.java:90)
16:30:41         at org.gradle.api.internal.tasks.testing.junitplatform.JUnitPlatformTestClassProcessor.stop(JUnitPlatformTestClassProcessor.java:85)
16:30:41         at org.gradle.api.internal.tasks.testing.SuiteTestClassProcessor.stop(SuiteTestClassProcessor.java:62)
16:30:41         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
16:30:41         at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
16:30:41         at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
16:30:41         at java.lang.reflect.Method.invoke(Method.java:498)
16:30:41         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:36)
16:30:41         at org.gradle.internal.dispatch.ReflectionDispatch.dispatch(ReflectionDispatch.java:24)
16:30:41         at org.gradle.internal.dispatch.ContextClassLoaderDispatch.dispatch(ContextClassLoaderDispatch.java:33)
16:30:41         at org.gradle.internal.dispatch.ProxyDispatchAdapter$DispatchingInvocationHandler.invoke(ProxyDispatchAdapter.java:94)
16:30:41         at com.sun.proxy.$Proxy2.stop(Unknown Source)
16:30:41         at org.gradle.api.internal.tasks.testing.worker.TestWorker$3.run(TestWorker.java:193)
16:30:41         at org.gradle.api.internal.tasks.testing.worker.TestWorker.executeAndMaintainThreadName(TestWorker.java:129)
16:30:41         at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:100)
16:30:41         at org.gradle.api.internal.tasks.testing.worker.TestWorker.execute(TestWorker.java:60)
16:30:41         at org.gradle.process.internal.worker.child.ActionExecutionWorker.execute(ActionExecutionWorker.java:56)
16:30:41         at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:113)
16:30:41         at org.gradle.process.internal.worker.child.SystemApplicationClassLoaderWorker.call(SystemApplicationClassLoaderWorker.java:65)
16:30:41         at worker.org.gradle.process.internal.worker.GradleWorkerMain.run(GradleWorkerMain.java:69)
16:30:41         at worker.org.gradle.process.internal.worker.GradleWorkerMain.main(GradleWorkerMain.java:74)
16:30:41 
16:30:41         Caused by:
16:30:41         org.awaitility.core.ConditionTimeoutException: Condition with lambda expression in org.apache.iceberg.spark.extensions.TestMerge that uses java.util.concurrent.atomic.AtomicInteger, java.util.concurrent.atomic.AtomicIntegerint was not fulfilled within 5 seconds.
16:30:41             at org.awaitility.core.ConditionAwaiter.await(ConditionAwaiter.java:167)
16:30:41             at org.awaitility.core.CallableCondition.await(CallableCondition.java:78)
16:30:41             at org.awaitility.core.CallableCondition.await(CallableCondition.java:26)
16:30:41             at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:985)
16:30:41             at org.awaitility.core.ConditionFactory.until(ConditionFactory.java:954)
16:30:41             at org.apache.iceberg.spark.extensions.TestMerge.lambda$testMergeWithSnapshotIsolation$19(TestMerge.java:1634)
16:30:41             at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
16:30:41             at java.util.concurrent.FutureTask.run(FutureTask.java:266)
16:30:41             at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
16:30:41             at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
16:30:41             at java.lang.Thread.run(Thread.java:748)

Willingness to contribute

  • I can contribute a fix for this bug independently
  • I would be willing to contribute a fix for this bug with guidance from the Iceberg community
  • I cannot contribute a fix for this bug at this time
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@manuzhang