niftynet.engine.handler_early_stopping module

This module implements an early stopping handler

class EarlyStopper(**_unused)[source]

Bases: object

This class handles iteration events to store the current performance as an attribute of the sender (i.e. application).

check_criteria(_sender, **msg)[source]

Printing iteration message with tf.logging interface. :param _sender: :param msg: an iteration message instance :return:

This function computes the generalisation loss as
Parameters:validation_his – performance history
Returns:generalisation loss
check_should_stop(performance_history, mode='mean', min_delta=0.03, kernel_size=5, k_splits=5)[source]

This function takes in a mode, performance_history and patience and returns True if the application should stop early. :param mode: {‘mean’, ‘robust_mean’, ‘median’, ‘generalisation_loss’,

‘median_smoothing’, ‘validation_up’} the default mode is ‘mean’
If your last loss is less than the average across the entire performance history stop training
Same as ‘mean’ but only loss values within 5th and 95th percentile are considered
As in mode=’mean’ but using the median
Computes generalisation loss over the performance history, and stops if it reaches an arbitrary threshold of 0.2.
This method check for performance increases in k sub-arrays of length s, where k x s = patience. Because we cannot guarantee patience to be divisible by both k and s, we define that k is either 4 or 5, depending on which has the smallest remainder when dividing.
Parameters:performance_history – a list of size patience with the performance

history :param min_delta: threshold for smoothness :param kernel_size: hyperparameter for median smoothing :param k_splits: number of splits if using ‘validation_up’ :return: