Objective

To create an image filter to transform the style of an image to the style of a particular painting using purely image processing techniques.

Motivation

Handling the above problem using convolutional neural networks has a huge processing overhead that we want to minimise so that more applications can effectively handle images

Neural Style Transfer

Refernce: Leon A. Gatys, Alexander S. Ecker,Matthias Bethge. “A Neural Algorithm of Artistic Style.”

NST, a part of Artistic Style Transfer, refers to a class of software algorithms that manipulate digital images, or videos, in order to adopt the appearance or visual style of another image. NST algorithms are characterized by their use of deep neural networks for the sake of image transformation. Popular use cases for NST are the creation of artificial artwork from photographs, for example by transferring the appearance of famous paintings to user-supplied photographs.

The main goal of this project was to explore Art-style-transfer through implementation. We’ll Implement a CNN model. FUture scope is to deploy it as a web app so that anyone can create stunning digital art which they could even sell as NFT’s.

Our Approach

It has been shown that a convolutional neural net algorithm can distinguish between the content of a piece of art and the style that it was painted in ([1] Leon Gatys). For our final project, we would like to explore this idea in the context of image processing. Specifically, we want to create an image filter that transfers the style of one painting to the content of another image. A clear motivation for finding a solution is for applications that handle many images at a time or for applications that cannot tolerate the training/learning time overhead that comes with the CNN (Michael Elad’s paper “Style-Transfer via Texture-Synthesis”).

We can see that the CNN approach can distinguish the style and the content of the images. We aim to do the same by retaining the objects of the image but using the style of the art.

We aimed to incorporate:

• Style/texture Synthesis: Migrating a style from a given image to the content of another, synthesizing a new image, which is an artistic mixture of the two.
• Style Fusion: Design an algorithm to follow broader structures of style at a higher level while giving it the freedom to make its own artistic decisions at a smaller scale.

Brief Steps:

• Pre-processing - Readying the image for processing
• Patch Matching - Finding patches of the image (content image) that are similar to patches of the artwork (style image)
• Style Synthesis
• Content Fusion
• Color Transfer
• Denoising

Future Work

• One clear area of improvement is making the algorithm robust to a greater range of style and content images.
• Patch matching can be optimized with principal component analysis (PCA) to reduce the computation time.
• Convolutional Neural Networks can also be implemented for Art Style Transfer. As per our research, we found it to be most commonly used approach for this purpose.

Detailed step-by-step process we carried out and results can be seen here: https://drive.google.com/file/d/1Y0IeQz_co9G1OjR3wL75UvNDvR_0YjEQ/view