Bug fixes & small additions

Changes:

New functionalities

• added method that computes Kullback-Leibler-Divergence
• ESN object can now be initialized by yaml config

New parallelization mode & generic loss computation

Changes:

Renaming

• renamed launch_process_RunturbESN to parallelize_seeds in study.py
• renamed Callback to callback_seeds in study.py
• renamed start_thread_RunturbESN to thread_run_turbESN in study.py
• renamed nstudy to nsettings in util.py, core.py & study.py
• renamed istudy to isetting in util.py, core.py & study.py
• renamed CreateStudyConfigArray to create_study_config_list in util.py
• renamed SaveStudy, ReadStudy, ReadMSE, ReadESNOutput to snake case equivalents
• renamed ComputeWassersteinDistance to compute_wasserstein_distance in util.py
• renamed ComputeMSE to compute_mse in util.py
• renamed ComputeR2 to compute_r2 in util.py
• renamed RunturbESN to run_turbESN in util.py
• renamed setDevice to set_device in core.py
• renamed toTorch to to_torch in core.py
• renamed _MSE_DEFAULT to _LOSS_DEFAULT in _modes.py

New functionalities

• added functionality to parallelize grid & random search setting over subprocesses (before: only random seeds were distributed among processors)

  • added parallelize_settings & callback_setting methods in study.py
  • choose between parallelization in run_gs.py via the parameter use_parallel_setting
  • this speeds up studies, where the no. settings is larger than the no. random seeds

• changed computation of mse in run_turbESN to more generic way: user uses dict esn.loss_func to specify which loss functions should be computed, results are returned in loss_dict

  • added None init. for esn.loss_func
  • adapted save_study, read_study in util.py
  • adapted read in core.py
  • renamed read_mse to read_loss in util.py
  • adapted read_loss in util.py
  • adapted thread_run_turbESN, callback_settings, callback_seeds in study.py
  • adapted forward_validate_auto_ESN in util.py (not tested)

Misc.

• fixed minor issues in run_gs.py
• fixed bug in plot_esn_predictions method in util.py
• added default values to style_dict in plot_esn_predictions & plot_activation_arg_distribution in util.py
• fixed error in read method in core.py, where self.xrows was not adapted correctly
• • added member variable u_pre_val to ESN object in core.py
• added option to set u_pre_val via SetValidationData in core.py
• use self.u_pre_val (if not None) in run_turbESN to initialize reservoir state before validation phase (make it more comparable to test phase, which is also preceded by state from GT data)
• self.val_init_input is now taken from self.u_pre_val (if val_init_input is None and u_pre_val is not None)
• removed iseed argument in read_loss, read_study, read_esn_output in util.py
• adpated basic_tour.ipynb notebook to new turbESN

ESN hyperparameter information & renaming

Changes:

Renaming

• changed variable names (stay more consistent in naming convention):

  • pred_init_input -> test_init_input (core.py)
  • x_fit -> x_train (util.py)
  • x_pred -> x_test (util.py)

• changed name user_study_w_config.py to run_gs.py

• changed name CreateHDF5Groups to create_hdf5_groups in util.py

• changed name InitRandomStudyOrder to init_random_search in util.py

• renamed doRandomSearch parameter to do_random_search in esn_config.yaml

• renamed verifyReservoirConvergence to verify_echo_state_property in core.py

• renamed activation_arg_dist method to compute_activation_arg_distribution in core.py

• renamed study_parameters to study_tuple in several methods in util.py

New functionalities

• added new fit option: pseudo-inverse
• added get_size method in core.py
• added plot_activation_arg_distribution & plot_esn_predictions methods to util.py
• added get_HP_info method in core.py
• added undo_minmax_scaling method in util.py
• added ReadESNOutput, ReadMSE to util.py (separated the ReadStudy method)

Refinements

• refined run_gs.py
• refined esn_config.yaml
• adapted read method in core.py to read all data from hdf5
• adapted compute_activation_arg_distribution method to feedback argument + added option to specify ESN prediction phase
• updated save method in core.py to work with hyperparameters.json
• updated init_random_search method in util.py to work with hyperparameters.json
• changed launch_thread_RunturbESN, launch_process_RunturbESN in study.py to assess whethe weight matrices Wres,Win,Wfb must be recomputed or can be computed once per random seed

Backend

• added _modes.py to root dir (.py files read constants & mode values from here now)
• use importlib.resources for finding path to hyperparameters.json
• added hyperparameters.json to root dir

Misc.

• fixed bug, that provided prediction input would not be used in predidct method in util.py
• added init_grid_search to util.py
• fixed bug in predict method in core.py
• added verify_echo_state_property to run_gs.py
• added fit_method & mode to hyperparameters.json
• fixed bug in read in core.py
• changed logging state in verify_echo_state_property method from error to warn in core.py

Feedback Weights & minor fixes

• added feedback weights
• removed small world network utility
• minor bug fixes
• leaking rate can no safely be specified as array (for each neuron)

Cross Validation

Changes

• changed identity matrix in fitting procedure to not include bias terms
• added option to run ESN w. third (validation) dataset (util.py & core.py)
• added function which checks user input data (prev. in RunturbESN)
• fixed problem in assert in RunturbESN in util.py
• adapted SaveStudy & ReadStudy to save mse_val & y_pred_val in util.py
• changed no. return variables in RunturbESN: now returns mse_train, mse_test, mse_val, y_pred_test, y_val
• added k fold forward walk validation procedure in util.py
• renamed LaunchThreads to launch_process_RunturbESN in study.py
• renamed LaunchSingleThread to launch_thread_RunturbESN in study.py
• adapted launch_thread_RunturbESN & Callback in study.py
• added launch_process_forward_validate_turbESN in study.py
• added launch_thread_forward_validate_turbESN in study.py
• added validationLength as ESN object value in core.py
• added validationLength to L63Reservoir in core.py
  added validationLength to ml4ablReservoir in core.py
• added cross_validation.py to turbESN
• added k fold forward walk validation data preparation to cross_validation.py
• added CrossValidation.save in cross_validation.py
• added option for single esn_id in CreateHDF5Groups in util.py
• adapted in ESN.save for single esn_id in core.py
• changed k fold forward walk validation return value to torch.tensors in util.py

minor changes & additional functions

• renamed some internal ESN variables
• added option of nonlinearity to training
• added Strogatz-Watts reservoir

first stable release

v0.0.1.5

Update README.md