Perceptual loss

The identification of content and style are core elements of a Neural Style Transfer (NST). The agreement of the content and style of two images is measured with the content_loss and style_loss, respectively.


In pystiche these losses are implemented Loss s. Loss s are differentiated between two types: RegularizationLoss and ComparisonLoss. A RegularizationLoss works without any context while a ComparisonLoss compares two images. Furthermore, pystiche differentiates between two different domains an Loss can work on: PixelOperator and EncodingOperator . A PixelOperator operates directly on the input_image while an EncodingOperator encodes it first.

In total pystiche supports four archetypes:


Builtin examples





Multi-layer encoder

One of the main improvements of NST compared to traditional approaches is that the agreement is not measured in the pixel or a handcrafted feature space, but rather in the learned feature space of a Convolutional Neural Network called encoder. Especially variants of the style_loss depend upon encodings, i. e. feature maps, from various layers of the encoder.

pystiche offers a MultiLayerEncoder that enables to extract all required encodings after a single forward pass. If the same operator should be applied to different layers of a MultiLayerEncoder, a MultiLayerEncodingLoss can be used.

Perceptual loss

The PerceptualLoss combines all Operator s in a single measure acting as joint optimization criterion. How the optimization is performed will be detailed in the next section.