Optimizing ComfyUI with Parallelism and Scaling: A Proposal to Use Ray

[tempwswm]

Feature Idea

I’ve noticed that ComfyUI currently executes tasks sequentially, which limits its efficiency for various use cases. Introducing proper parallelism could lead to significant improvements:

• For personal use, better parallelism would increase processing efficiency.
• For rented GPUs, parallel execution can save costs by maximizing resource utilization.
• For large cluster service providers, parallelism is essential to handle complex workloads effectively.

To address this, I suggest considering Ray as the backbone for scaling and parallel execution in ComfyUI. Here are the reasons why Ray is a strong candidate:

Why Use Ray for Scaling in ComfyUI?

1. Ray is purpose-built for scaling AI and Python applications.
   It’s widely adopted in the AI community and has proven scalability features for both single-node and distributed applications.

2. Ray’s objects system supports lazy evaluation and result caching.
   This can enhance execution efficiency for tasks that produce reusable outputs, such as certain node computations.

3. Similar paradigms: ComfyUI uses ‘nodes,’ and Ray uses ‘actors.’
   This similarity makes ComfyUI's workflow easier to adapt to Ray's model, thus simplifying integration.

4. Ray supports multiple isolated virtual environments.
   This feature is especially useful for resolving library/environment conflicts. Currently, ComfyUI manages different custom nodes that could rely on varying versions of the same library. While these do not necessarily cause runtime errors, they often lead to frustrating deprecation warnings. Ray’s isolation capabilities can mitigate this.

5. Community demand for better parallelism and distribution
   As mentioned earlier, the community needs improved parallelism. With Ray, it becomes feasible to distribute tasks across multiple GPUs, CPUs, or even clusters.

If any part of my explanation is unclear, feel free to reach out! I used a translator, so I’d be happy to discuss and refine further if needed. 😊

Existing Solutions

#162
#3538

Other

Key Resources / Links

ray github
ray core doc
SD example

By the way, it might also be worth considering UV for installing custom nodes. UV is increasingly becoming the de facto standard for managing Python dependencies. Integrating UV could provide a cleaner and more efficient method for handling custom node installations.

[ltdrdata]

I think it would be best to discuss this in the discussion section.

[senlyu163]

Very interesting perspective! Does anyone have relevant experience?

[tempwswm]

I have, and am looking at the comfy source code.
But I'm not good enough to write pr anytime soon, that's why I'm proposing this, I need assistance😆

[cb88]

Xdit apparently also uses Ray... so if some major stuff is getting written it probably should also take into consideration adding advanced parallelism features xdit is developing.