New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: add tcd sampler #3370
base: master
Are you sure you want to change the base?
feat: add tcd sampler #3370
Conversation
@JettHu it seems to be just a sampler in this PR, I don't see TCD scheduler included - am I missing something? Isn't TCD scheduler necessary too, for TCD method to work as intended? I've tried TCD scheduler available as custom node from here: https://github.com/dfl/comfyui-tcd-scheduler, and I've got different results from sgm_uniform. This PR + TCD scheduler (custom node): Test workflow: UPDATE: vs And for higher number of steps (and eta) results seem to be getting similar: vs So... it seems TCD sampler with sgm_uniform scheduler should do. |
This came up when trying to us this PR while the original repo works fine, oddly. (Am several commits behind though..) |
Sorry, it was my mistake. this PR does contain only a sampler. Because it's called scheduler in diffusers, I'm used to it. So the title of this PR should be called tcd sampler, which is more reasonable.
TCD and LCM use the same scheduler, which are I have not seen the implementation details of comfyui-tcd-scheduler. As mentioned here, my implementation ComfyUI-TCD may be more consistent with the original TCD. The content of this PR basically comes from my repo comfyui-tcd ComfyUI-TCD. |
@rabidcopy Do you mind sharing your workflow and comfyui startup parameters? I updated a version to ensure that the indices in sample_tcd are on the cpu. |
Thanks for the update. The PR seems to work now without throwing any errors. I was using the workflow from here changed to use the node from this PR instead of your dedicated node for it. I think the device issue might have stemmed from my current usage of the DirectML backend as I am currently on Windows with an AMD GPU. I think the reason the dedicated node didn't have this issue for me is due to this line right here. https://github.com/JettHu/ComfyUI-TCD/blob/main/__init__.py#L61 But that's just a guess. Edit: Also for further context, I was inferencing an SDXL model that was loaded in low VRAM mode. |
From the link you provided: |
@RandomGitUser321 This PR does not require an additional The upper part of the screenshot below shows setting If necessary, I can adjust the PR and rename |
Oh I wasn't talking about your version, yours seems to work fine. I was talking about the dlf version. There are two versions of the comfyui implementation, yours https://github.com/JettHu/ComfyUI-TCD and his https://github.com/dfl/comfyui-tcd-scheduler |
Oh sorry, misunderstood |
There's a low chance this gets merged. If you look at the commit history, Comfy hardly ever merges pull requests that aren't bug fixes. I'm not complaining, this is their repo that works well for me after all. |
That's a bit of a pity |
This one is getting merged once I check that it matches the reference code. |
adapted code from comfyanonymous/ComfyUI#3370 TODO: check if virtual scheduler tcd is needed for using sampling_base ModelSamplingDiscreteDistilled or if it's better to use sgm_uniform directly without patching
@comfyanonymous It works pretty fine for me - via SamplerTCD EulerA with a bit higher gamma: SDXL + TCD workflow, original prompt.json UPDATE: normal EulerA seems to do pretty well (maybe even a bit better?) with TCD LoRA, too: |
@mhh0318 @jabir-zheng @JettHu Could you guys please look into that difference mentioned by @comfyanonymous (between original diffusers and ComfyUI implementation), and double-check if sampler part is implemented properly? UPDATE: at the moment I am not sure how much of myself being impressed with TCD was based on TCD sampler / scheduler, or it was just TCD LoRA. I am getting nice results from this LoRA when simply using it with sgm_uniform scheduler and EulerA sampler. |
This PR solved #2985 . Add Trajectory Consistency Distillation Support.
sample_tcd
.alphas_cumprod
tocalculate_denoised
&noise_scaling
, so I add register_buffer alphas_cumprod in ModelSamplingDiscrete.tcd
sampling incomfy_extras.nodes_model_advanced.ModelSamplingDiscrete
SamplerTCD
node incomfy_extras.nodes_custom_sampler
for seteta
(referred to asgamma
in the paper).The upper part of the screenshot below shows setting
eta
(gamma
in paper) value throughSamplerTCD
. In the lower part, selectingtcd
directly inKSampler
will use the default 0.3 asgamma
.If someone are in a hurry, you can use my plug-in repository.