niftynet.engine.handler_early_stopping module

This module implements an early stopping handler

class EarlyStopper(**_unused)

Bases: object

This class handles iteration events to store the current performance as an attribute of the sender (i.e. application).

check_criteria(_sender, **msg)

Printing iteration message with tf.logging interface. :param _sender: :param msg: an iteration message instance :return:

compute_generalisation_loss(validation_his)

This function computes the generalisation loss as

l[-1]-min(l)/max(l)-min(l)

Parameters:validation_his – performance history Returns:generalisation loss

check_should_stop(performance_history, mode='mean', min_delta=0.03, kernel_size=5, k_splits=5)

This function takes in a mode, performance_history and patience and returns True if the application should stop early. :param mode: {‘mean’, ‘robust_mean’, ‘median’, ‘generalisation_loss’,

‘median_smoothing’, ‘validation_up’} the default mode is ‘mean’

mean:

If your last loss is less than the average across the entire performance history stop training

robust_mean:

Same as ‘mean’ but only loss values within 5th and 95th percentile are considered

median:

As in mode=’mean’ but using the median

generalisation_loss:

Computes generalisation loss over the performance history, and stops if it reaches an arbitrary threshold of 0.2.

validation_up:

This method check for performance increases in k sub-arrays of length s, where k x s = patience. Because we cannot guarantee patience to be divisible by both k and s, we define that k is either 4 or 5, depending on which has the smallest remainder when dividing.

Parameters:performance_history – a list of size patience with the performance

history :param min_delta: threshold for smoothness :param kernel_size: hyperparameter for median smoothing :param k_splits: number of splits if using ‘validation_up’ :return: