niftynet.layer.crf module

Re-implementation of [1] in Tensorflow for volumetric image processing.

[1] Zheng et al. “Conditional random fields as recurrent neural networks.” ICCV 2015.

class CRFAsRNNLayer(alpha=5.0, beta=5.0, gamma=5.0, T=5, aspect_ratio=None, mu_init=None, w_init=None, name='crf_as_rnn')[source]

Bases: niftynet.layer.base_layer.TrainableLayer

This class defines a layer implementing CRFAsRNN described in [1] using a bilateral and a spatial kernel as in [2]. Essentially, this layer smooths its input based on a distance in a feature space comprising spatial and feature dimensions. High-dimensional Gaussian filtering adapted from [3].

[1] Zheng et al., [2] Krahenbuhl and Koltun, [3] Adam et al.,

__init__(alpha=5.0, beta=5.0, gamma=5.0, T=5, aspect_ratio=None, mu_init=None, w_init=None, name='crf_as_rnn')[source]

Currently this layer supports spatial ND dense CRF with CPU only. To place the layer on CPU:

with tf.device('/cpu:0'):
    crf_layer = CRFAsRNNLayer()
    crf_output = crf_layer(features, raw_logits)

To ensure backpropagations during training are placed on CPU as well, the optimiser should be used with argument colocate_gradients_with_ops=True, e.g.,:

train_op = tf.train.GradientDescentOptimizer(.5).minimise(
    training_loss, colocate_gradients_with_ops=True)
  • alpha – bandwidth for spatial coordinates in bilateral kernel. Higher values cause more spatial blurring
  • beta – bandwidth for feature coordinates in bilateral kernel Higher values cause more feature blurring
  • gamma – bandwidth for spatial coordinates in spatial kernel Higher values cause more spatial blurring
  • T – number of stacked layers in the RNN
  • aspect_ratio – spacing of adjacent voxels (allows isotropic spatial smoothing when voxels are not isotropic)
  • mu_init – initial compatibility matrix [n_classes x n_classes] default value: -1.0 * eye(n_classes)
  • w_init

    initial kernel weights [2 x n_classes] where w_init[0] are the weights for the bilateral kernel,

    w_init[1] are the weights for the spatial kernel.

    default value: [ones(n_classes), ones(n_classes)]

  • name
layer_op(I, U)[source]

Compute T iterations of mean field update given a dense CRF.

This layer maintains trainable CRF model parameters (a compatibility function and m kernel weights).

  • I – feature maps used in the dense pairwise term of CRF
  • U – activation maps used in the unary term of CRF (before softmax)

Maximum a posteriori labeling (before softmax)

ftheta(U, H1, permutohedrals, mu, kernel_weights, norms, name)[source]

A mean-field update

  • U – the unary potentials (before softmax)
  • H1 – the previous mean-field approximation to be updated
  • permutohedrals – fixed position vectors for fast filtering
  • mu – compatibility function
  • kernel_weights – weights bilateral/spatial kernels
  • norms – precomputed normalisation factor
  • name – layer name

updated mean-field distribution


Embedding the position vectors in a high-dimensional space, the lattice points are stored in hash tables.

The function computes: - translation by the nearest reminder-0 - ranking permutation to the canonical simplex - barycentric weights in the canonical simplex

Parameters:position_vectors – N x d position
Returns:barycentric weights, blur neighbours points in the hyperplane
permutohedral_compute(data_vectors, barycentric, blur_neighbours1, blur_neighbours2, indices, name, reverse)[source]

Splat, Gaussian blur, and slice

  • data_vectors – value map to be filtered
  • barycentric – embedding coordinates
  • blur_neighbours1 – first neighbours’ coordinates relative to indices
  • blur_neighbours2 – second neighbours’ coordinates relative to indices
  • indices – corresponding locations of data_vectors
  • name – layer name
  • reverse – transpose the Gaussian kernel if True

filtered data_vectors (sliced to the original space)