New features and feature requests #87
Replies: 14 comments 13 replies
-
not sure what you mean by that exactly, but sounds interesting ;-) can you elaborate further? excited about further config-options for blending etc., too! well, you know what's on @anxiousottergames ' and my wishlist.. ;-p but like I already said: you focus on what you deem necessary, first. it's coming along nicely :-) |
Beta Was this translation helpful? Give feedback.
-
Sounds good to me. I'm not sure if you had them listed in order of importance but I personally feel like even the order of the planned features listed is perfect. |
Beta Was this translation helpful? Give feedback.
-
With Roop development now shut down, this is the only place where we can expect improvements on this nice tool. After a lot of tests, I figured out (at least within my test results) that the biggest flip side is the eye movement. It is totally missing in a video swap. The model seems to be looking straight forward, irrespective of the actions. Hence, the model looks like visually impaired or blind. This is not noticeable in a still image swap because it is a still image, there is no action. But in a video, this lack of eye movement is very easily noticeable. You can tell within 4-5 seconds that the video is fake. Above all of the other enhancements discussed, I think we first need to focus on how to copy the eye-movements of the target to the output while we swap the face from the source. Basically, the face will come from the source, we need to detect the eyeballs, copy the eyeball direction from the target (not using the source eyeball directions). That will add life to our end product. This technology is already there because Adobe Photoshop can do this. We can change eye direction in a photo using Photoshop. We have to do that same thing with the output image after detecting the direction from the target frames. But I am not sure if there is an open source project already available for this. A close match would be Gaze Correction, but it has limited features. It may be used as a start point. |
Beta Was this translation helpful? Give feedback.
-
That's why I linked https://github.com/ai-forever/ghost which especially cared about gaze and emotions. From my limited testing it did this better than the rest but the face mask was lacking or rather looked too artificial. |
Beta Was this translation helpful? Give feedback.
-
can we get a fps changer (probably doable with ffmpeg even without reencoding) this is useful for some high resolution videos that are 60fps and because of this takes a lot of time but can be 30fps cutting processing time in half , maybe processing every second frame instead of every frame ? Also like in stable diffusion extension can we select faces by number they appear on screen ? (1 and 3 for example out of 4 faces) Both of these are especially useful for 180vr videos where a persons face appear two times but since they are in different angles app doesnt recognize both as the same almost always. This is not a problem if only one person is in the video just select all faces BUT if there are more than one person on screen we either need to change all faces or just one persons face on one side of the video. |
Beta Was this translation helpful? Give feedback.
-
I tested Fast Segment-Anything yesterday and am quite disappointed. It's faster than the current Clip2Seg but surprisingly very bad when using low-resolution inputs. If it can identify objects, the resulting masks are way better though. I don't know if it's worth the hassle. |
Beta Was this translation helpful? Give feedback.
-
Suggestion on the Side: What about an Upload Source for an Audio File (mp3) for "directly" a new Lip Sync? -> Overwrits the Lip Sync of the Target Video and Change it with the "Manuellextra located Audio File. :-) |
Beta Was this translation helpful? Give feedback.
-
Controlnet?Sent from my iPhoneOn Aug 21, 2023, at 6:24 AM, JeetGuhaThakurta ***@***.***> wrote:
With Roop development now shut down, this is the only place where we can expect improvements on this nice tool. After a lot of tests, I figured out (at least within my test results) that the biggest flip side is the eye movement. It is totally missing in a video swap. The model seems to be looking straight forward, irrespective of the actions. Hence, the model looks like visually impaired or blind.
This is not noticeable in a still image swap because it is a still image, there is no action. But in a video, this lack of eye movement is very easily noticeable. You can tell within 4-5 seconds that the video is fake. Above all of the other enhancements discussed, I think we first need to focus on how to copy the eye-movements of the target to the output while we swap the face from the source.
Basically, the face will come from the source, we need to detect the eyeballs, copy the eyeball direction from the target (not using the source eyeball directions). That will add life to our end product. This technology is already there because Adobe Photoshop can do this. We can change eye direction in a photo using Photoshop. We have to do that same thing with the output image after detecting the direction from the target frames. But I am not sure if there is an open source project already available for this. A close match would be Gaze Correction, but it has limited features. It may be used as a start point.
—Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>
|
Beta Was this translation helpful? Give feedback.
-
I'm searching for the possibility to erode and blur the face mask but on the last roop-unleashed (3.3.4) I can't find the options, is that possible? I'm asking that 'cause i'm still obtain a slight ghost-box around my swapped face and i want to improve the results blurring the box a bit. |
Beta Was this translation helpful? Give feedback.
-
Since I had a potato computer, I integrated gpen 256 thinking it was better than nothing. But I couldn't get the Original/Enhanced image blend ratio part to work correctly. It only works between 0.90 and 1. When it goes below that, it adds strange blurs and distorts the image. what would be the reason ? |
Beta Was this translation helpful? Give feedback.
-
@C0untFloyd quick question. I noticed the TEMP folder gets really chunky after extended use and keeps facesets duplicated from the UI. Can i periodically delete the contents of the temp folder? |
Beta Was this translation helpful? Give feedback.
-
Transfered here from the discussion started by @aripsam Since this version is pretty stable and has been running well for a while now, I'm just wondering if there are any plans to add any more features? Anything in the pipeline? Some features I can think of -
|
Beta Was this translation helpful? Give feedback.
-
i also have a suggestion, maybe its caused by its limitation? |
Beta Was this translation helpful? Give feedback.
-
Btw. I forked a new promising hairstyle transfer repo today, which would be a great addition to roop unleashed once it is working: I can't get it to work so far though, not in Colab and not on my machine. Perhaps one of you can enlighten me, the problem is the dynamic compilation of python modules which I can't seem to get right. Also there is a lot of hopefully unnecessary Jupyter stuff in it, this would have to be removed for our own local needs. |
Beta Was this translation helpful? Give feedback.
-
Here I would like to collect and show planned features for roop unleashed.
My wishlist so far would be:
Beta Was this translation helpful? Give feedback.
All reactions