Skip to content

Latest commit

 

History

History
58 lines (43 loc) · 3.63 KB

README.md

File metadata and controls

58 lines (43 loc) · 3.63 KB

SciIE (Under Construction)

This repository contains code and models for replicating results from the following publication:

Part of the codebase is extended from lsgn and e2e-coref.

Requirements

  • Python 2.7
    • TensorFlow 1.8.0
    • pyhocon (for parsing the configurations)
    • tensorflow_hub (for loading ELMo)

Getting Started

  • Python 2.7

  • TensorFlow 1.8.0

  • pyhocon (for parsing the configurations)

  • tensorflow_hub (for ELMo)

  • GloVe embeddings and downloading data:
    ./scripts/fetch_required_data.sh

  • Build kernels: ./scripts/build_custom_kernels.sh (Please make adjustments to the script based on your OS/gcc version)

Setting up for ELMo (if you use your own data)

  • Some of our models are trained with the ELMo embeddings. We use the ELMo model loaded by tensorflow_hub.
  • It is recommended to cache ELMo embeddings for training and validating efficiency. Please modify the corresponding filenames and run python generate_elmo.py to generate ELMo embeddings.

Training Instructions

  • Experiment configurations are found in experiments.conf

  • The parameter main_metrics can be selected from coref, ner, relation or any combination of the three, such as coref_ner_relation which indicates the F1 score for averaged F1 score for coref, ner and relation. The model is tuned and saved based on the resulting averaged F1 score.

  • The parameters ner_weight, coref_weight and relation_weight are weights for the multi-task objective. If set the weight to 0 then the task is not trained.

  • Choose an experiment that you would like to run, e.g. scientific_best_ner

  • For a single-machine experiment, run the following two commands in parallel:

    • python singleton.py <experiment>
    • python evaluator.py <experiment>

  • Results are stored in the logs directory and can be viewed via TensorBoard.

  • For final evaluation of the checkpoint with the maximum dev F1:

    • python test_single.py <experiment>

Other Quirks

  • It does not use GPUs by default. Instead, it looks for the GPU environment variable, which the code treats as shorthand for CUDA_VISIBLE_DEVICES.
  • The evaluator should not be run on GPUs, since evaluating full documents does not fit within GPU memory constraints.
  • The training runs indefinitely and needs to be terminated manually.

Making Predictions with Pretrained Models

  • Define the output path in experiments.conf as output_path, the system will output the results of eval_path to output_path. The output file is also a json file, which has thesame format as eval_path. Then run python write_single.py <experiment>

Best Models