You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Is your feature request related to a problem? Please describe.
Currently, there is no estimate shown for how long the currently executing node is expected to take.
Describe the solution you'd like
For some node types, this would be difficult to impossible to calculate, however for other node types it would be trivial to calculate.
Some nodes seem to vary in how long they take for each chunk.
However, most nodes seem to follow a pattern where the variance in times taken for each chunk is rather small. (ie. the time taken for a chunk is generally close to the average of all the chunks)
For these types of nodes, you could calculate the remaining time with average(chunk_durations) * chunk_count - execution_time
Perhaps a new method, timeRemaining() could be introduced on the Node class which returns an expected completion time or None if it cannot be estimated for that node type.
The default implementation would return None, and if the time can be reasonably calculated for a node, it would be overridden with the calculation.
This would also allow for calculating the remaining time for nodes where the time for each chunk is known to follow an exponential curve or a linear function, for example.
Of course, this would not calculate the expected execution time for the entire pipeline as that would be significantly more difficult, but it can at least give you an estimation for the current node. It would however require that at least 1 chunk have been completed for the currently executing node.
Describe alternatives you've considered
This can be calculated manually, however it is cumbersome to do. It would be helpful to instead have it presented in the UI.
The text was updated successfully, but these errors were encountered:
Is your feature request related to a problem? Please describe.
Currently, there is no estimate shown for how long the currently executing node is expected to take.
Describe the solution you'd like
For some node types, this would be difficult to impossible to calculate, however for other node types it would be trivial to calculate.
Some nodes seem to vary in how long they take for each chunk.
However, most nodes seem to follow a pattern where the variance in times taken for each chunk is rather small. (ie. the time taken for a chunk is generally close to the average of all the chunks)
For these types of nodes, you could calculate the remaining time with
average(chunk_durations) * chunk_count - execution_time
Perhaps a new method,
timeRemaining()
could be introduced on theNode
class which returns an expected completion time orNone
if it cannot be estimated for that node type.The default implementation would return
None
, and if the time can be reasonably calculated for a node, it would be overridden with the calculation.This would also allow for calculating the remaining time for nodes where the time for each chunk is known to follow an exponential curve or a linear function, for example.
Of course, this would not calculate the expected execution time for the entire pipeline as that would be significantly more difficult, but it can at least give you an estimation for the current node. It would however require that at least 1 chunk have been completed for the currently executing node.
Describe alternatives you've considered
This can be calculated manually, however it is cumbersome to do. It would be helpful to instead have it presented in the UI.
The text was updated successfully, but these errors were encountered: