GPU Benchmark! 4070 4080 4090 3080 3090

[metamountain]

Could we Do a benchmark for GPU´s ?
Need to get new hardware .
To keep it simple we could use the default load worklow on 1024*1024 with sdxl 1.0.
Seed 1
I know that ram comes in play when workflows get more complex , but for the start simple ist best.
Please post the second run. first run ist a little bit corupted by loading data.

theory pudget:

toms hardware 1111:

[metamountain]

2070 rtx mobile
100%|█████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:19<00:00, 1.05it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 24.22 seconds

[epaulekas]

Nvidia RTX 4090 FE

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:03<00:00, 6.17it/s]
Prompt executed in 3.61 seconds

(edit: used default resolution first go around)

[Robson1970]

CPU vs GPU 1024x1024 2 steps

Intel 13900KS 24,25 sec.
Nvidia 4090 0,42 sec.

https://files.catbox.moe/f4wb53.mp4

CPU.vs.GPU.test_2.mp4

[metamountain]

Not quite a benchmark, you Gorilla.

[metamountain]

RTX 3090
100%|████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.61it/s]
Prompt executed in 6.16 seconds

[JorgeR81]

Just for laughs ...
GTX 1070
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [01:03<00:00, 3.16s/it]
Prompt executed in 72.44 seconds

[BrechtCorbeel]

My 1070 is my favorite, was my longtime friend still in good use, pretty good for what it was, 3060 is 2 faster with only half as many additional cores, but with the 4090 I have now, uncomparable.

[TheMadHatt3r]

RTX 3070 OC Drivers:~528.xx

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:08<00:00, 2.26it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 10.89 seconds

[YulienPohl]

My poor old GPU 😢
GTX 1050 Ti
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [02:51<00:00, 8.55s/it]
Prompt executed in 202.83 seconds

[ltdrdata]

RTX 4080
(Ubuntu 22.04, pytorch 2.2.0)

got prompt
100%|██████████| 20/20 [00:05<00:00,  3.53it/s]
Prompt executed in 6.46 seconds

[TheMadHatt3r]

Would love to get some 3060 / 4060 data to better complete the regression... Then I'll post proposed for every 3000/4000 card.
Some older 2000 could be helpful to understand how the smaller VRAM scales as well.

[ShamanFlamingo]

RTX 3090 driver:551.61

RTX 3090 driver:531.18

[ecpknymt]

RTX 4060ti 16GB

[TheMadHatt3r]

RTX 4070 12Gb

got prompt
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 3.21it/s]
Prompt executed in 7.13 seconds

[askmyteapot]

WSL2 (6.1.x kernel)
Pytorch 2.3.0 from source
Xformers recent build (0.25)

3060Ti

Requested to load SDXL
Loading 1 new model
100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:09<00:00,  2.05it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 13.29 seconds

Tesla P40

got prompt
100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:45<00:00,  2.28s/it]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 49.26 seconds

No xformers

3060ti

Requested to load SDXL
Loading 1 new model
100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:09<00:00,  2.11it/s]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 13.00 seconds

Tesla P40

got prompt
100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:45<00:00,  2.27s/it]
Requested to load AutoencoderKL
Loading 1 new model
Prompt executed in 48.97 seconds

[stepahin]

4090 FE, Ryzen 7900X, Samsung 980 2TB M.2
Win 11, ComfyUI cu121

got prompt
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 7.61it/s]
Prompt executed in 3.08 seconds

[RysDawid]

RTX 3070 Laptop GPU
Asus ZenBook Duo

Loading 1 new model 100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:11<00:00, 1.73it/s] Requested to load AutoencoderKL Loading 1 new model Prompt executed in 15.44 seconds

[fangpop]

Rtx 4090 16 GB mobile Dell
13th Gen Intel(R) Core(TM) i9-13980HX
64 GB
model weight dtype torch.float16, manual cast: None
model_type EPS
Using pytorch attention in VAE
Using pytorch attention in VAE
Requested to load SDXLClipModel
Loading 1 new model
loaded completely 0.0 1560.802734375 True
Requested to load SDXL
Loading 1 new model
loaded completely 0.0 4897.0483474731445 True
100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:05<00:00, 3.70it/s]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
Prompt executed in 16.92 seconds

[Rogala]

This is where the pain and sadness
PC: CPU (Intel i3-12100f), GPU (NVIDIA GTX 1650 SUPER 4GB), RAM (DDR4-3600 32GB), NVMe (KINGSTON SNV2S1000G 1TB), SWAP 40GB, OS Win 10 ComfyUI v0.2.3 Release pytorch version: 2.5.0+cu124
Set vram state to: LOW_VRAM

100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [02:28<00:00, 7.45s/it]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 319.11416244506836 True
Prompt executed in 158.67 seconds

[aw-leigh]

MSI GeForce RTX 3080 VENTUS 3X 10G / Ryzen 7 7800X3D / 32GB RAM
Python version: 3.11.9 / pytorch version: 2.5.0+cu124

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:06<00:00, 3.00it/s]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
Prompt executed in 11.85 seconds

[FeepingCreature]

Sapphire Nitro+ Radeon RX 7900 XTX Vapor-X with AMD Go Fast extension and some other tweaks, Linux 6.11.5, ROCM 6.2.0:

got prompt
100%|[...]| 20/20 [00:04<00:00,  4.21it/s]
Prompt executed in 5.35 seconds

[FeepingCreature]

Update: I have it at 5.3it/s now with some very conservative undervolting and a few more tweaks.

[Dreamcolor]

T_T
This is mine.
HP ZBOOK Power G7 Mobile Workstation
CPU: Intel(R) Core(TM) i5-10300H CPU @ 2.50GHz 2.50 GHz
RAM: 64.0 GB
GPU: Nvidia Quadro P620 (4G VRAM)
SSD: 512G + 1T

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [04:37<00:00, 13.88s/it]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 319.11416244506836 True
Warning: Ran out of memory when regular VAE decoding, retrying with tiled VAE decoding.
Prompt executed in 332.66 seconds

[theoldbar]

Hello all 😊, finally signed after a few months of following the discussions, now i can annoy everyone with questions lol

results from my Acer TC1760 , i5-12400, 32gb DDR4 3200 ram, 1tb WD black sn770 nvme, and no laughing .. RTX 3050 6gb.
the 3050 was the best choice with the proprietary 300w psu. Anyway to the results ...

Total VRAM 6144 MB, total RAM 32612 MB
pytorch version: 2.5.1+cu124
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3050 : cudaMallocAsync
Using pytorch cross attention

100%|██████████████████████████████████████████████████████████████████████████████████| 20/20 [00:25<00:00, 1.29s/it]
Requested to load AutoencoderKL
Loading 1 new model
loaded completely 0.0 159.55708122253418 True
Prompt executed in 29.09 seconds

[pluberd]

RTX 3090, NVIDIA-SMI 550.120, Driver Version: 550.120, CUDA Version: 12.4

The Model is already loaded. First run after cooling down the RTX to 33°C is always slower than the second run. After that it is getting slower until around 73°C.

Temp: 33°C, with "Save Image"

Prompt executed in 5.37 seconds
Prompt executed in 5.32 seconds

Temp: 33°C, with "Preview Image"

Prompt executed in 5.28 seconds
Prompt executed in 5.25 seconds

Temp: 73°C, with "Save Image"

Prompt executed in 5.46 seconds
Prompt executed in 5.46 seconds

Temp: 73°C, with "Preview Image"

Prompt executed in 5.39 seconds
Prompt executed in 5.39 seconds

[bigcat88]

We recently re-ran our ComfyUI-based flows and thought the results might be of interest to the local community. 🛠️📊

Here's just a screenshot:

If you're interested in other graphs, follow the link: Hardware Results

Also one page down (Hardware Results Raw) - there are VRAM measurements(benchmarks were mainly written for this).

[FeepingCreature]

Are you running with AMD Go Fast? /~https://github.com/Beinsezii/comfyui-amd-go-fast It's basically a free 2x.

[bigcat88]

No, I didn't even know about such a thing.

Thanks for notifying me that it exists, I need to check it.

In 3-4 weeks there will be new benchmarks, much more clearly grouped, with tests of token encoders when the prompt changes, and I will add some video model.

[FeepingCreature]

Ah, also note that if you're installing this today, it's probably easiest to grab one of the prebuilt wheels from Beinsezii/comfyui-amd-go-fast#2 (comment) because AMD is and continues to be a shitshow.

[Fluffinko]

win11 wsl2 ubuntu
Total VRAM 24576 MB, total RAM 90510 MB
pytorch version: 2.5.1+cu118
xformers version: 0.0.28.post3
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 3090 : cudaMallocAsync
Using xformers cross attention

100%|███████████████████████████████████████████████████████████████████████████████████| 20/20 [00:04<00:00, 4.03it/s]
Prompt executed in 5.49 seconds

[to-sora]

got prompt
100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 20/20 [00:02<00:00, 7.10it/s]
Prompt executed in 3.22 seconds ( include text encoder , ensured no cache)
4090D , linux mint , 7950X 16-Core, 128 , 1024 * 1024 , SDXL seed 1

[Hangover3832]

Total VRAM 16376 MB, total RAM 65461 MB
pytorch version: 2.3.1+cu121
WARNING[XFORMERS]: xFormers can't load C++/CUDA extensions. xFormers was built for:
    PyTorch 2.2.2+cu121 with CUDA 1201 (you have 2.3.1+cu121)
    Python  3.11.8 (you have 3.11.6)
  Please reinstall xformers (see /~https://github.com/facebookresearch/xformers#installing-xformers)
  Memory-efficient attention, SwiGLU, sparse and more won't be available.
  Set XFORMERS_MORE_DETAILS=1 for more details
xformers version: 0.0.25.post1
Set vram state to: NORMAL_VRAM
Device: cuda:0 NVIDIA GeForce RTX 4080 : cudaMallocAsync
Using pytorch attention

100%|█████████████████████| 20/20 [00:03<00:00,  5.10it/s]
Prompt executed in 4.50 seconds

[mcpdigital]

RTX 4070 Ti Super
Total VRAM 16376 MB, total RAM 81806 MB
pytorch version: 2.5.1+cu121
Python 3.12.4
NVIDIA Driver: 566.36
CPU: AMD Ryzen 7 5700X3D 8-Core Processor - Arch: AMD64 - OS: Windows 11

4.8 seconds,
4.7 seconds
4.9 seconds

[zarushi]

RTX 3060 12GB
AMD Ryzen 5 5600X / 64GB / Win 11 / Python 3.10.11 / PyTorch 2.6.0 / CUDA 12.6
100%|███████████████████████████████████████████| 20/20 [00:13<00:00, 1.46it/s]
Prompt executed in 14.92 seconds

[bigcat88]

Made new benchmarks on a more recent version of ComfyUI and PyTorch 2.6(for AMD 2.7 nightly was used)

This is already the second series of benchs, we tried to take into account the shortcomings of the first benchmarks and made that for each Task a new prompt is given, so that the text encoder is also tested.

Similar benchmark suites were also removed, we tried to use 30 steps for generation everywhere
(if this is not a lighting version of the model)

Full interactive results link for those who is interested

[FeepingCreature]

If you're getting 8s on 7900 XTX SDXL default, it sounds like AMD Go Fast didn't work out for you?

[bigcat88]

Go Fast made Flux and SD3.5 faster

But then I updated ROCM, Go Fast break, and in Comfy landed a commit with PyTorch cross-attention support, and I stayed with it(it does not affect SDXL, but makes faster DIT models, the same as Go Fast).

And we can't add Go Fast support because there is no convenient installation option for different versions of Rocm/PyTorch and different versions of Python.

note: aesthetic it is a playground-v2.5-1024px-aesthetic model

[FeepingCreature]

Yeah... fucking ROCM. AMD really make it as hard as possible to help them not suck.

[cool9-MK]

RTX 5070ti, overclocked (+7% roughly).

100%█████████████████████████| 20/20 [00:04<00:00, 4.46it/s]
Prompt executed in 4.91 seconds