Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(katana-tasks): separate tasks spawning responsibility from TaskManager #2514

Merged
merged 2 commits into from
Oct 10, 2024

Conversation

kariy
Copy link
Member

@kariy kariy commented Oct 10, 2024

the idea is to separate task management into two components; TaskManager and TaskSpawner. TaskSpawner is derived from a TaskManager and thus can only be created from a TaskManager and is used to spawn any tasks to its associated manager.

the issue that prompted this refactor is when due to the Drop implementation of TaskManager, where it will cancel all ongoing tasks. because TaskManager was cloned around, the underlying cancellation token is the same across all instances of TaskManager, and thus when one instance is dropped, it promptly cancelled all tasks of all manager.

Summary by CodeRabbit

  • New Features

    • Enhanced logging configuration for improved debugging.
    • New functions for clearer output of genesis state information during server startup.
    • Introduction of a new task spawner for better task management and execution.
  • Bug Fixes

    • Improved error handling and logging for pipeline failures.
  • Documentation

    • Updated comments and documentation to reflect new features and changes.
  • Refactor

    • Structural improvements to task management and encapsulation for better maintainability.

Copy link

coderabbitai bot commented Oct 10, 2024

Walkthrough

Ohayo! This pull request introduces several modifications primarily focused on the NodeArgs struct and task management within the Katana project. Key updates include changes to logging configurations, the addition of new functions for genesis state output, and a shift in task management logic to improve maintainability. Method signatures across various modules have been updated, and new structs and methods have been introduced to enhance task spawning capabilities. Overall, these changes aim to refine the functionality and clarity of the codebase.

Changes

File Path Change Summary
bin/katana/src/cli/node.rs Updated NodeArgs struct, logging configuration, added print_genesis_contracts and print_genesis_accounts functions, and modified execute method.
crates/katana/node/src/lib.rs Transitioned from self.task_manager.clone() to self.task_manager.task_spawner() in Node and LaunchedNode. Updated method signatures for launch and spawn.
crates/katana/node-bindings/src/lib.rs Introduced Katana and KatanaInstance structs, added methods for configuration and lifecycle management of Katana instances, and enhanced error handling.
crates/katana/pipeline/Cargo.toml Added tokio as a workspace dependency.
crates/katana/pipeline/src/lib.rs Updated logging in Pipeline struct's run method and added error logging in into_future.
crates/katana/pipeline/src/stage/sequencing.rs Replaced task_manager with task_spawner, updated return types for run_messaging and run_block_production, and modified execute method for task monitoring.
crates/katana/tasks/src/lib.rs Introduced TokioTaskSpawner struct for task management and modified BlockingTaskPool.
crates/katana/tasks/src/manager.rs Encapsulated TaskManager state within Inner struct, updated methods for task spawning.
crates/katana/tasks/src/task.rs Updated TaskBuilder to use TaskSpawner instead of TaskManager, modified constructor and methods.

Possibly related PRs

Suggested reviewers

  • glihm: Suggested as a reviewer for their expertise in the relevant areas of the codebase.

🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (12)
crates/katana/pipeline/src/lib.rs (1)

49-52: Excellent logging enhancements, sensei!

The modifications to the logging statements are spot-on:

  1. Specifying the "pipeline" target for the stage execution log improves log categorization.
  2. Adding a period at the end of the message enhances readability.
  3. The new log message for pipeline completion provides better visibility into the execution flow.

These changes will make our logs more informative and easier to parse. Dojo approved!

For even better consistency, consider adding the stage ID to the completion log as well:

- info!(target: "pipeline", "Pipeline finished.");
+ info!(target: "pipeline", "Pipeline finished. Total stages: {}.", self.stages.len());
bin/katana/src/cli/node.rs (2)

Line range hint 441-484: Sugoi work on enhancing the print_intro function, sensei!

The addition of JSON logging and the new helper functions print_genesis_contracts and print_genesis_accounts greatly improve the clarity and structure of the genesis state output. This change makes it easier for users and automated tools to parse and understand the initial state of the blockchain.

One small suggestion for consistency:

Consider using the serde_json::to_string_pretty function for the JSON output to make it more readable when logged. Here's a quick example:

- info!(
-     target: LOG_TARGET,
-     "{}",
-     serde_json::json!({
-         "accounts": accounts.map(|a| serde_json::json!(a)).collect::<Vec<_>>(),
-         "seed": format!("{}", seed),
-         "address": format!("{address}"),
-     })
- )
+ info!(
+     target: LOG_TARGET,
+     "{}",
+     serde_json::to_string_pretty(&serde_json::json!({
+         "accounts": accounts.map(|a| serde_json::json!(a)).collect::<Vec<_>>(),
+         "seed": format!("{}", seed),
+         "address": format!("{address}"),
+     })).unwrap()
+ )

This will make the JSON output more human-readable when logged.


Line range hint 486-557: Domo arigato for the new helper functions, sensei!

The print_genesis_contracts and print_genesis_accounts functions greatly improve the clarity and structure of the genesis state output. The tabular format makes it easy for users to understand the initial state of the blockchain.

One small suggestion for consistency:

Consider using a consistent style for displaying addresses and hashes. Currently, some are displayed with the {:#064x} format specifier, while others use the default {} format. For consistency, you might want to use the same format for all addresses and hashes. Here's an example:

- println!(
-     r"
- | Contract        | Fee Token
- | Address         | {}
- | Class Hash      | {:#064x}",
-     genesis.fee_token.address, genesis.fee_token.class_hash,
- );
+ println!(
+     r"
+ | Contract        | Fee Token
+ | Address         | {:#064x}
+ | Class Hash      | {:#064x}",
+     genesis.fee_token.address, genesis.fee_token.class_hash,
+ );

Apply this change throughout both functions for a consistent display of all addresses and hashes.

crates/katana/pipeline/src/stage/sequencing.rs (1)

50-51: Ohayo, sensei! Consider simplifying task handling when messaging_config is None.

Currently, when messaging_config is None, a task is spawned that waits indefinitely on future::pending::<()>(). You might consider not spawning a task in this case or handling it differently to avoid unnecessary resource usage.

crates/katana/tasks/src/task.rs (2)

88-88: Ohayo sensei, please ensure task name is correctly logged even when name is None

Since name can be None, consider handling this in the log statement to avoid logging task = None.


130-131: Consider handling None task names more explicitly, sensei

When the task name is None, using an empty string may make logs less informative. Consider providing a default name or indicating when a task name is not provided.

crates/katana/tasks/src/lib.rs (4)

25-25: Reminder: Replace with TaskSpawner in manager.rs

Ohayo sensei! There's a TODO comment indicating that this code should be replaced with TaskSpawner in manager.rs. Would you like me to assist in implementing this change or open a GitHub issue to track it?


26-28: Correct grammatical errors in documentation comments

Ohayo sensei! I noticed some typos in the documentation:

  • "This is simple wrapper" should be "This is a simple wrapper".
  • "a tokio runtime" should be "the Tokio runtime".
  • "a tokio's runtime" should be "Tokio's runtime".

Apply this diff to fix the typos:

 /// A task spawner for spawning tasks on a tokio runtime. This is simple wrapper around a tokio's
-/// runtime [Handle] to easily spawn tasks on the runtime.
+/// runtime [Handle] to easily spawn tasks on the runtime.
+///
+/// This is a simple wrapper around the Tokio runtime. It uses Tokio's
+/// runtime [Handle] to easily spawn tasks on the runtime.

Line range hint 35-58: Consider merging multiple impl blocks for TokioTaskSpawner

Ohayo sensei! I noticed that TokioTaskSpawner has multiple impl blocks. For better readability and maintainability, consider merging them into a single impl block.


Line range hint 77-79: Fix typo in documentation comment

Ohayo sensei! There's a small typo in the documentation: "spawing" should be "spawning".

Apply this diff to correct the typo:

 /// For spawing blocking IO-bound tasks, use [TokioTaskSpawner::spawn_blocking] instead.
-/// For spawing blocking IO-bound tasks, use [TokioTaskSpawner::spawn_blocking] instead.
+/// For spawning blocking IO-bound tasks, use [TokioTaskSpawner::spawn_blocking] instead.
crates/katana/tasks/src/manager.rs (2)

211-211: Accessing internal fields in tests may break encapsulation.

In the test, assert_eq!(manager.inner.tracker.len(), 3); directly accesses manager.inner.tracker. To maintain encapsulation, consider adding a public method or accessor in TaskManager to expose the required information for testing purposes.


217-217: Ohayo, sensei! Prefer using public methods to check cancellation status in tests.

The assertion !manager.inner.on_cancel.is_cancelled() accesses the on_cancel token directly. For better encapsulation, add a method to TaskManager like is_cancelled() that returns the cancellation status, and use it in the test.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 7cd7d08 and 0d77872.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (8)
  • bin/katana/src/cli/node.rs (1 hunks)
  • crates/katana/node/src/lib.rs (2 hunks)
  • crates/katana/pipeline/Cargo.toml (1 hunks)
  • crates/katana/pipeline/src/lib.rs (3 hunks)
  • crates/katana/pipeline/src/stage/sequencing.rs (4 hunks)
  • crates/katana/tasks/src/lib.rs (1 hunks)
  • crates/katana/tasks/src/manager.rs (8 hunks)
  • crates/katana/tasks/src/task.rs (6 hunks)
🧰 Additional context used
🔇 Additional comments (24)
crates/katana/pipeline/Cargo.toml (1)

18-18: Ohayo! Nice addition of Tokio, sensei!

The inclusion of Tokio as a workspace dependency is a solid choice. This asynchronous runtime will likely enhance our task management capabilities, aligning perfectly with our mission to separate task spawning responsibilities.

crates/katana/pipeline/src/lib.rs (3)

9-9: Ohayo, sensei! Nice addition to the tracing imports!

Adding the error macro alongside info is a great move. It enhances our error handling capabilities, allowing us to log errors more effectively. This change aligns well with the new error logging implemented later in the file.


62-66: Ohayo! Impressive error handling upgrade, sensei!

The addition of inspect_err in the into_future implementation is a masterful touch:

  1. It provides specific error logging for pipeline failures, enhancing our ability to debug issues.
  2. The use of inspect_err is a clean and efficient way to add logging without altering the future's result.
  3. Including the error details in the log message will be invaluable for troubleshooting.

This change significantly improves the robustness of our pipeline execution. Well done!


Line range hint 1-87: Ohayo, sensei! Overall assessment of the Pipeline enhancements

The changes made to the Pipeline struct and its associated methods are excellent. They significantly improve logging and error handling, which aligns perfectly with the PR objectives of enhancing task management. Here's a summary of the improvements:

  1. Enhanced import statements for better tracing capabilities.
  2. Improved logging in the run method for clearer execution tracking.
  3. Robust error handling in the into_future implementation.

These modifications will make the pipeline more maintainable, easier to debug, and more informative during execution. Great work on refining the Katana project's core functionality!

bin/katana/src/cli/node.rs (1)

259-262: Ohayo! Nice touch on the logging configuration, sensei!

The updated default log filter now includes "tasks=debug" at the beginning, which aligns well with the PR's focus on enhancing task management. This change will provide more detailed logs for task-related operations, making it easier to debug and monitor the new task spawning system.

crates/katana/pipeline/src/stage/sequencing.rs (6)

11-12: Ohayo, sensei! Imports for task handling are correctly added.

The addition of TaskHandle and TaskSpawner imports is appropriate for the new task spawning functionality.


22-22: Field task_spawner correctly replaces task_manager.

Updating the struct to use task_spawner aligns with the refactoring goals to separate task spawning responsibilities.


31-35: Constructor updated to accept TaskSpawner.

The new function now correctly accepts task_spawner and initializes the struct accordingly.


38-51: run_messaging method correctly returns TaskHandle<()>.

The method now returns TaskHandle<()>, allowing for proper task management and monitoring. The use of task_spawner to spawn the MessagingTask is well-implemented.


55-61: run_block_production method appropriately spawns the block production task.

Changing run_block_production to return TaskHandle<()> and using task_spawner to spawn the task aligns with the refactoring objectives.


73-90: Execution logic effectively manages task lifecycles.

The execute method now uses tokio::select! to monitor both messaging and block_production tasks. This ensures that if either task finishes unexpectedly, an error is logged, and appropriate actions can be taken.

crates/katana/tasks/src/task.rs (5)

36-36: Ohayo sensei! Updating TaskBuilder to use TaskSpawner is appropriate

This change aligns with the refactoring objectives to separate task spawning responsibilities.


80-80: Ohayo! Updated destructuring of self is appropriate, sensei

This change correctly reflects the updated fields in TaskBuilder.


85-85: Adjusted cancellation token retrieval from spawner is appropriate

This modification aligns with the refactoring to use TaskSpawner, sensei.


103-103: Update to use spawner.spawn(fut) is appropriate, sensei

This change correctly uses the TaskSpawner to spawn the future.


48-49: Ensure TaskBuilder::new is updated throughout the codebase, sensei

Since TaskBuilder::new now takes TaskSpawner instead of TaskManager, please make sure all instantiations are updated accordingly.

Run the following script to verify all calls to TaskBuilder::new use TaskSpawner:

crates/katana/tasks/src/manager.rs (6)

65-67: Ohayo, sensei! Great job on introducing TaskSpawner for better separation of concerns.

The new method task_spawner and the TaskSpawner struct effectively separate task spawning responsibilities from TaskManager. This aligns well with the PR objectives to resolve issues related to shared cancellation tokens and enhances the modularity of the task management system.


145-145: Ensure thread safety when tracking futures across spawners.

Using self.inner.tracker.track_future(task); within TaskSpawner is appropriate. Confirm that TaskTracker is thread-safe and can handle concurrent tracking from multiple TaskSpawner instances without any race conditions or data inconsistencies.


169-170: Ohayo, sensei! Confirm the cancellation behavior in TaskManager's Drop implementation.

Calling self.inner.on_cancel.cancel(); in the Drop implementation ensures that all ongoing tasks are cancelled when the TaskManager is dropped. Verify that this behavior is intentional and that dropping TaskManager prematurely won't lead to unintended task cancellations elsewhere in the system.


245-245: Ohayo, sensei! Verify that tasks respect the graceful shutdown signal.

When spawning a task with graceful_shutdown(), ensure that all other tasks are cancelled appropriately and that the shutdown process waits for the graceful task to complete. Confirm that resources are cleaned up correctly and no tasks linger after shutdown.


261-261: Ensure panics in critical tasks trigger the intended shutdown behavior.

In critical_task_graceful_shutdown_on_panicked, a critical task intentionally panics. Verify that the TaskManager handles this panic by initiating a graceful shutdown without causing unwanted side effects or leaving tasks in an inconsistent state.

Run the following script to confirm panic handling:


31-31: Ohayo, sensei! Verify the removal of the Clone trait from TaskManager.

The Clone trait has been removed from TaskManager (#[derive(Debug, Clone)]#[derive(Debug)]). Ensure that no parts of the codebase rely on cloning TaskManager. If cloning is still required elsewhere, consider whether TaskSpawner, which implements Clone, can serve as a replacement.

Run the following script to check for any cloning of TaskManager:

✅ Verification successful

Ohayo, sensei! It seems the initial check didn't find any cloning of TaskManager, but to be sure, let's dive a bit deeper. 🧐


Ohayo, sensei! 🎉
Cloning of TaskManager has been successfully removed, and TaskSpawner is now being used as its replacement throughout the codebase. All checks confirm that no parts of the code rely on cloning TaskManager.

🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Search for usages of `TaskManager` cloning.
# Expected result: There should be no instances where `TaskManager` is cloned.

rg 'TaskManager.*\.clone\(\)'

Length of output: 31


Script:

#!/bin/bash
# Description: Search for manual implementations of the `Clone` trait for `TaskManager`.
# Expected result: There should be no manual `Clone` implementations for `TaskManager`.

rg 'impl\s+Clone\s+for\s+TaskManager'

Length of output: 39


Script:

#!/bin/bash
# Description: Search for any indirect cloning of `TaskManager` through other structs or wrappers.
# Expected result: There should be no indirect cloning of `TaskManager`.

rg '.*clone\(\).*TaskManager'

Length of output: 31


Script:

#!/bin/bash
# Description: Verify if `TaskSpawner` is used as a replacement for cloning `TaskManager`.
# Expected result: Instances where `TaskSpawner` is used instead of cloning `TaskManager`.

rg 'TaskSpawner'

Length of output: 3272

crates/katana/node/src/lib.rs (2)

135-135: Ohayo sensei! Good use of task_spawner() for improved task management.

Switching to self.task_manager.task_spawner() enhances the separation of responsibilities by delegating task spawning to the TaskSpawner. This aligns perfectly with the refactoring goals and improves the code's clarity.


145-150: Ohayo sensei! Task spawning with builder pattern enhances configurability.

Utilizing the builder pattern with build_task().critical().name("Pipeline").spawn(pipeline.into_future()) increases the configurability and readability of task spawning. This approach makes it clear which tasks are critical and aids in monitoring and debugging.

crates/katana/tasks/src/manager.rs Show resolved Hide resolved
@kariy
Copy link
Member Author

kariy commented Oct 10, 2024

tests failing caused by not displaying the actual port address that has been binded by the rpc server. but instead display the raw port specified from the config. this is an issue in the case where port number is set to 0 (ie when requesting for random port). as the node-bindings relies on the address displayed in the startup log. fixed in #2518

@kariy kariy force-pushed the katana/tasks-spawner branch from 0d77872 to 7858a28 Compare October 10, 2024 19:12
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (8)
crates/katana/tasks/src/task.rs (1)

Line range hint 1-165: Ohayo! Overall refactoring looks stellar, sensei!

The changes consistently implement the separation of task spawning responsibility from TaskManager to TaskSpawner. This new structure should effectively resolve the issue with unintended task cancellations when a TaskManager instance is dropped.

The addition of debug logging for tasks with graceful shutdown enhances observability, which is a nice touch.

One minor suggestion to consider:

Consider adding a brief comment explaining the purpose of TaskSpawner and its relationship to TaskManager at the top of the file. This would provide helpful context for future developers working on this code.

crates/katana/tasks/src/manager.rs (2)

51-57: Ohayo! Nice implementation of the new structure, sensei!

The new method correctly initializes the inner field with an Arc<Inner>, and the task_spawner method provides a clean way to create a TaskSpawner. These changes align well with the refactoring objectives.

A small suggestion: Consider adding a comment in the task_spawner method to explain that it creates a new TaskSpawner instance each time it's called, if that's the intended behavior.

Also applies to: 65-67


111-136: Ohayo! Excellent introduction of TaskSpawner, sensei!

The new TaskSpawner struct and its methods align perfectly with the PR objectives. It provides a clean API for task management and spawning, separating these responsibilities from TaskManager.

A small suggestion: Consider adding a brief comment to the cancellation_token method explaining when and why a user might need to access it directly.

crates/katana/node/src/lib.rs (1)

145-150: Ohayo, pipeline sensei! Impressive task spawning upgrade.

The new implementation for spawning the pipeline task is a significant improvement. It provides more granular control over the task creation process, allowing for properties like criticality and name to be set. This aligns well with the PR's goal of enhancing task management.

One small suggestion to consider:

Consider extracting the task name "Pipeline" into a constant at the top of the file. This would improve maintainability if the name needs to be referenced elsewhere or changed in the future. For example:

const PIPELINE_TASK_NAME: &str = "Pipeline";

// Then use it in the task spawning:
.name(PIPELINE_TASK_NAME)

What do you think, sensei? Would you like me to prepare a small PR for this change?

crates/katana/tasks/src/lib.rs (4)

25-25: Ohayo! Remember to address the TODO comment

There's a TODO indicating that TokioTaskSpawner should be replaced with TaskSpawner in manager.rs. To keep the codebase clean and prevent technical debt, please consider resolving this.

Do you need assistance implementing this change, or would you like me to open a new GitHub issue to track this task, sensei?


Line range hint 27-29: Correct grammatical errors in documentation

In the documentation for TokioTaskSpawner, there are some grammatical errors:

  • "This is simple wrapper around a tokio's runtime [Handle]" should be "This is a simple wrapper around Tokio's runtime [Handle]".

Apply this diff to fix the grammatical errors:

-/// A task spawner for spawning tasks on a tokio runtime. This is simple wrapper around a tokio's
-/// runtime [Handle] to easily spawn tasks on the runtime.
+/// A task spawner for spawning tasks on a Tokio runtime. This is a simple wrapper around Tokio's
+/// runtime [Handle] to easily spawn tasks on the runtime.

Line range hint 37-54: Consider merging multiple impl blocks for TokioTaskSpawner

Currently, TokioTaskSpawner has two separate impl blocks. While Rust allows multiple impl blocks, merging them can improve readability and maintainability.


Line range hint 78-83: Fix typo in documentation

In the BlockingTaskPool documentation, there's a typo:

  • "For spawing blocking IO-bound tasks..." should be "For spawning blocking IO-bound tasks..."

Apply this diff to correct the typo:

-/// CPU-bound tasks. For spawing blocking IO-bound tasks, use [TokioTaskSpawner::spawn_blocking]
+/// CPU-bound tasks. For spawning blocking IO-bound tasks, use [TokioTaskSpawner::spawn_blocking]
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Files that changed from the base of the PR and between 0d77872 and 7858a28.

⛔ Files ignored due to path filters (1)
  • Cargo.lock is excluded by !**/*.lock
📒 Files selected for processing (9)
  • bin/katana/src/cli/node.rs (1 hunks)
  • crates/katana/node-bindings/src/lib.rs (1 hunks)
  • crates/katana/node/src/lib.rs (2 hunks)
  • crates/katana/pipeline/Cargo.toml (1 hunks)
  • crates/katana/pipeline/src/lib.rs (3 hunks)
  • crates/katana/pipeline/src/stage/sequencing.rs (4 hunks)
  • crates/katana/tasks/src/lib.rs (1 hunks)
  • crates/katana/tasks/src/manager.rs (8 hunks)
  • crates/katana/tasks/src/task.rs (6 hunks)
🚧 Files skipped from review as they are similar to previous changes (4)
  • bin/katana/src/cli/node.rs
  • crates/katana/pipeline/Cargo.toml
  • crates/katana/pipeline/src/lib.rs
  • crates/katana/pipeline/src/stage/sequencing.rs
🧰 Additional context used
🔇 Additional comments (13)
crates/katana/tasks/src/task.rs (4)

9-9: Ohayo! New imports look good, sensei!

The addition of debug from tracing and TaskSpawner aligns well with the refactoring objectives. These imports support the new logging functionality and the transition from TaskManager to TaskSpawner.

Also applies to: 12-12


36-36: TaskBuilder update looks sharp, sensei!

The replacement of TaskManager with TaskSpawner in the TaskBuilder struct and its new method aligns perfectly with the refactoring goals. This change effectively separates the task spawning responsibility, enhancing the overall architecture.

Also applies to: 48-49


80-80: Ohayo! TaskBuilder::spawn method is looking mighty fine, sensei!

The updates to the spawn method are spot-on:

  1. Using spawner instead of manager maintains consistency with the new architecture.
  2. Obtaining the cancellation token from spawner aligns with the refactoring goals.
  3. The new debug logging enhances observability for tasks with graceful shutdown.
  4. Task spawning through spawner completes the transition to the new structure.

These changes effectively implement the separation of task spawning responsibility.

Also applies to: 85-85, 88-88, 103-103


130-131: CriticalTaskBuilder changes are on point, sensei!

The updates to CriticalTaskBuilder are well-executed:

  1. Separating the task name retrieval enhances code readability.
  2. Obtaining the cancellation token from spawner aligns perfectly with the new TaskSpawner structure.

These modifications successfully integrate the CriticalTaskBuilder with the refactored task management system.

crates/katana/tasks/src/manager.rs (6)

22-30: Ohayo! Documentation update looks great, sensei!

The updated documentation for TaskManager clearly explains the new task spawning mechanism using TaskSpawner. This change aligns well with the refactoring objectives and provides clear guidance for users.


31-34: Excellent refactoring, sensei!

The TaskManager struct has been streamlined to use an Arc<Inner>, which should improve shared access to the manager's state. The removal of the Clone derive is appropriate since Arc already provides the necessary sharing mechanism.


36-45: Ohayo! Great job on improving encapsulation, sensei!

The introduction of the Inner struct is a solid design choice. It nicely encapsulates the internal state of TaskManager, separating it from the public interface. This should make the code more maintainable and easier to reason about.


Line range hint 72-108: Ohayo! Consistent updates to methods, sensei!

The changes to wait_for_shutdown, shutdown, handle, and wait methods are well-implemented. They correctly access the fields through the new inner struct, maintaining proper encapsulation. This consistency helps in keeping the code clean and easy to understand.


145-146: Ohayo! Nice update to task tracking, sensei!

The change in the spawn_inner method to use self.inner.tracker for tracking the task is spot on. It ensures that tasks are properly managed using the encapsulated tracker, maintaining consistency with the new structure.


169-170: Ohayo! Great updates to Drop and tests, sensei!

The addition of a trace log in the Drop implementation for TaskManager is a nice touch for debugging. The test cases have been correctly updated to use the new TaskSpawner API, which is crucial for ensuring the refactored code works as expected.

Regarding the past comment about potential hangs in the task_with_graceful_shutdown test:

The infinite loops in the spawned tasks still don't check for cancellation, which could potentially lead to hangs if the shutdown mechanism fails. Consider adding cancellation checks within these loops to ensure graceful termination.

You could modify the loops like this:

spawner.build_task().spawn(async {
    loop {
+       if spawner.cancellation_token().is_cancelled() {
+           break;
+       }
        time::sleep(Duration::from_secs(1)).await;
    }
});

This change would make the tests more robust and prevent potential hangs.

Also applies to: 204-261

crates/katana/node/src/lib.rs (2)

135-137: Ohayo, task management sensei! Excellent refactor.

The change from self.task_manager.clone() to self.task_manager.task_spawner() is a great improvement. It aligns perfectly with the PR objective of separating task spawning responsibilities. This focused approach to task management should lead to better control over task lifecycles and improved overall system stability.


Line range hint 1-450: Ohayo, Katana sensei! Overall assessment of the refactor.

The changes implemented in this file successfully address the main objectives of the PR. The separation of task spawning responsibilities from TaskManager to TaskSpawner is well executed. This refactoring should resolve the issue with task cancellation mentioned in the PR objectives, where dropping a TaskManager instance was causing unintended cancellations of tasks in other managers.

The new implementation provides better control over task creation and management, which should lead to improved stability and maintainability of the system. The changes are consistent and well-integrated into the existing codebase.

Great work, sensei! This refactor is a significant step forward for the Katana project.

crates/katana/node-bindings/src/lib.rs (1)

625-625: Ohayo, sensei! The code looks good here.

Initializing Katana with Katana::new().spawn(); is appropriate and correctly spawns the instance with default settings.

Comment on lines +632 to +634
// try to connect as a provider
let provider = JsonRpcClient::new(HttpTransport::new(dbg!(katana.endpoint_url())));
assert!(provider.chain_id().await.is_ok())
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue

Ohayo, sensei! Consider removing the dbg! macro for cleaner test output.

The dbg! macro is useful for debugging but can clutter test outputs with unnecessary information. It's best to remove it before merging to keep the test output clean.

Apply this diff to remove the dbg! macro:

-let provider = JsonRpcClient::new(HttpTransport::new(dbg!(katana.endpoint_url())));
+let provider = JsonRpcClient::new(HttpTransport::new(katana.endpoint_url()));
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
// try to connect as a provider
let provider = JsonRpcClient::new(HttpTransport::new(dbg!(katana.endpoint_url())));
assert!(provider.chain_id().await.is_ok())
// try to connect as a provider
let provider = JsonRpcClient::new(HttpTransport::new(katana.endpoint_url()));
assert!(provider.chain_id().await.is_ok())

@kariy kariy merged commit 9a1e300 into main Oct 10, 2024
13 checks passed
@kariy kariy deleted the katana/tasks-spawner branch October 10, 2024 20:07
steebchen added a commit that referenced this pull request Oct 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant