niftynet.layer.crf module

Re-implementation of [1] in Tensorflow for volumetric image processing.

[1] Zheng et al. “Conditional random fields as recurrent neural networks.” ICCV 2015. https://arxiv.org/abs/1502.03240

class CRFAsRNNLayer(alpha=5.0, beta=5.0, gamma=5.0, T=5, aspect_ratio=None, mu_init=None, w_init=None, name='crf_as_rnn')

Bases: niftynet.layer.base_layer.TrainableLayer

This class defines a layer implementing CRFAsRNN described in [1] using a bilateral and a spatial kernel as in [2]. Essentially, this layer smooths its input based on a distance in a feature space comprising spatial and feature dimensions. High-dimensional Gaussian filtering adapted from [3].

[1] Zheng et al., https://arxiv.org/abs/1502.03240 [2] Krahenbuhl and Koltun, https://arxiv.org/pdf/1210.5644.pdf [3] Adam et al., https://graphics.stanford.edu/papers/permutohedral/

__init__(alpha=5.0, beta=5.0, gamma=5.0, T=5, aspect_ratio=None, mu_init=None, w_init=None, name='crf_as_rnn')

Currently this layer supports spatial ND dense CRF with CPU only. To place the layer on CPU:

with tf.device('/cpu:0'):
    crf_layer = CRFAsRNNLayer()
    crf_output = crf_layer(features, raw_logits)

To ensure backpropagations during training are placed on CPU as well, the optimiser should be used with argument colocate_gradients_with_ops=True, e.g.,:

train_op = tf.train.GradientDescentOptimizer(.5).minimise(
    training_loss, colocate_gradients_with_ops=True)

Parameters:

• alpha – bandwidth for spatial coordinates in bilateral kernel. Higher values cause more spatial blurring
• beta – bandwidth for feature coordinates in bilateral kernel Higher values cause more feature blurring
• gamma – bandwidth for spatial coordinates in spatial kernel Higher values cause more spatial blurring
• T – number of stacked layers in the RNN
• aspect_ratio – spacing of adjacent voxels (allows isotropic spatial smoothing when voxels are not isotropic)
• mu_init – initial compatibility matrix [n_classes x n_classes] default value: -1.0 * eye(n_classes)
• w_init –

  initial kernel weights [2 x n_classes] where w_init[0] are the weights for the bilateral kernel,

  w_init[1] are the weights for the spatial kernel.

  default value: [ones(n_classes), ones(n_classes)]

• name –

layer_op(I, U)

Compute T iterations of mean field update given a dense CRF.

This layer maintains trainable CRF model parameters (a compatibility function and m kernel weights).

Parameters:

• I – feature maps used in the dense pairwise term of CRF
• U – activation maps used in the unary term of CRF (before softmax)

Returns:

Maximum a posteriori labeling (before softmax)

ftheta(U, H1, permutohedrals, mu, kernel_weights, norms, name)

A mean-field update

Parameters:

• U – the unary potentials (before softmax)
• H1 – the previous mean-field approximation to be updated
• permutohedrals – fixed position vectors for fast filtering
• mu – compatibility function
• kernel_weights – weights bilateral/spatial kernels
• norms – precomputed normalisation factor
• name – layer name

Returns:

updated mean-field distribution

permutohedral_prepare(position_vectors)

Embedding the position vectors in a high-dimensional space, the lattice points are stored in hash tables.

The function computes: - translation by the nearest reminder-0 - ranking permutation to the canonical simplex - barycentric weights in the canonical simplex

Parameters:position_vectors – N x d position Returns:barycentric weights, blur neighbours points in the hyperplane

permutohedral_compute(data_vectors, barycentric, blur_neighbours1, blur_neighbours2, indices, name, reverse)

Splat, Gaussian blur, and slice

Parameters:

• data_vectors – value map to be filtered
• barycentric – embedding coordinates
• blur_neighbours1 – first neighbours’ coordinates relative to indices
• blur_neighbours2 – second neighbours’ coordinates relative to indices
• indices – corresponding locations of data_vectors
• name – layer name
• reverse – transpose the Gaussian kernel if True

Returns:

filtered data_vectors (sliced to the original space)