The other day I saw an article on the Verge talking about a lip sync deepfake technique that anyone could do using the wave2lip algorithm on GitHub. This technique uses a Google Collab notebook and all of the computing happens in the cloud. Often when I see a tutorial about deep learning or artificial intelligence image manipulation I get excited and want to try it only to be foiled by its complexity. I usually find that I am missing code dependencies and have to install various python libraries. Once I get those libraries installed I then find out that I don’t have a powerful enough GPU or other necessary computing hardware. This technique uses the Collab notebook provided by the repository creator and allows anyone to have fun creating lip-synced deepfaked videos.. I think it’s important for as many people as possible to understand this technology because it will become more and more prevalent and we need to have media literacy to know what could potentially be real and what is no. We need an informed public that can debate about future legislation or needed regulation about these technologies as they become more powerful. For now I think that deep learning has lots of potential creative uses for animation and other visual communication.
Although there are lots of potentially bad uses for this technology there are just as many creative uses for making animated drawings and other creative visual content. Be good.
Link to GitHub Repository:
https://github.com/Rudrabha/Wav2Lip
Link to Google Collab:
https://colab.research.google.com/drive/1tZpDWXz49W6wDcTprANRGLo2D_EbD5J8?usp=sharing
0:00 – Introduction
1:20 – Setup folders in Google Drive
1:40 _ Wav2Lip GitHub Repository
1:59 – Download Pre-trained Model
2:15 – Google Collab Notebook
3:00 – Upload Pre-trained Model to Google Drive
3:18 – Uninstall Tensorflow GPU
3:48 – Upload Video and Audio files
4:07 – Rename files in Collab Notebook
4:30 – Run Wav2Lip on video
4:59 – Download lip synced video
5:12 – Variations to try
6:04 – Lip sync a painting
6:30 – Conclusion