#Model suggestions for restoring old movies

6 messages · Page 1 of 1 (latest)

muted moss
#

What models should I look at to restore / colorize / upscale old super-8 movies (by myself, don't want product recommendations)? I'm happy to code it up, to slice it by hand into frames and reassemble, don't need UI 🙂

I found these so far https://github.com/microsoft/Bringing-Old-Photos-Back-to-Life/ and https://github.com/jantic/DeOldify

GitHub
GitHub - microsoft/Bringing-Old-Photos-Back-to-Life: Bringing Old P...

Bringing Old Photo Back to Life (CVPR 2020 oral). Contribute to microsoft/Bringing-Old-Photos-Back-to-Life development by creating an account on GitHub.

GitHub
GitHub - jantic/DeOldify: A Deep Learning based project for coloriz...

A Deep Learning based project for colorizing and restoring old images (and video!) - GitHub - jantic/DeOldify: A Deep Learning based project for colorizing and restoring old images (and video!)

deft iris
#

For upscaling, consider image super resolution models. there is the SR3 model which uses diffusion (https://iterative-refinement.github.io/) for upscaling OR the SRGAN/ESRGAN models (https://arxiv.org/abs/1809.00219) which use a GAN approach to image upscaling/super resolution. There are pretrained ESRGAN models available both in Huggingface Hub (https://huggingface.co/models?sort=downloads&search=esrgan) and Tensorflow Hub (https://www.tensorflow.org/hub/tutorials/image_enhancing) depending on your preference.

SR3: Image Super-Resolution via Iterative Refinement
arXiv.org
ESRGAN: Enhanced Super-Resolution Generative Adversarial Networks

The Super-Resolution Generative Adversarial Network (SRGAN) is a seminal work
that is capable of generating realistic textures during single image
super-resolution. However, the hallucinated details are often accompanied with
unpleasant artifacts. To further enhance the visual quality, we thoroughly
study three key components of SRGAN - network ...

Models - Hugging Face
TensorFlow
Image Super Resolution using ESRGAN  |  TensorFlow Hub
muted moss
#

Thanks. Anything for colorization and restoration (can be of just images, not neccesarily videos)

deft iris
#

You covered the models I can think of for that (specifically deoldify)

muted moss
#

thanks, sounds like I got myself a pipeline to try then.

muted moss
#

FYI tried couple of different models, but found nothing interesting. BOPBtL managed to remove some scratches, as well as legitimate parts of the picture, like railings or wires 🙂 DeOldify produced washed out colors, the original colors are better (no surprise here).

Also tried VRT and RVRT, but also nothing special. Tried upscaling using LDSR (latent diffusion upscaler bundled with stable-diffusion-webui), and it produced somewhat good output... but not much better than a plain unsharp mask (and it takes ages to run). SwinIR_4x upscaler was way faster, but produced plastic-looking images. ESRGAN wasn't very interesting either.

Looks like I'd need to train my own models, most of these models look like they are research-only, and are made to prove a point, but not to actually work.