The two popular open source packages DeepFaceLab (DFL) and FaceSwap entered the public arena in 2017. Martin Anderson argues at Metaphysic that since their inception, improvements in deepfake quality have pushed the original code to its outermost limits.
In 2020 a method of recreating objects by stitching together multiple viewpoint photos inside a neural network emerged. The original NeRF paper by Mildenhall et al. (ECCV 2020) feeds a fully connected network a 3D position plus a viewing direction and gets back density and view-dependent colour, which classic volume rendering turns into new images. Given a limited number of viewpoints, NeRF calculates the ‘missing views’ by recognizing shapes, textures, transparency, and lighting values, and estimating and synthesizing the views that aren’t present in the source data. Rendering used to be slow, but Google Research’s SNeRG “bakes” a trained NeRF into a sparse voxel grid that renders at more than 30 frames per second on a laptop GPU. NeRF could challenge the state of the art while remaining in the open source arena.