Biomedical word sense disambiguation with bidirectional long short-term memory and attention-based neural networks

Abstract

In this paper, we propose two deep-learning-based models for supervised WSD - a model based on bi-directional long short-term memory (BiLSTM) network, and an attention model based on self-attention architecture. Our result shows that the BiLSTM neural network model with a suitable upper layer structure performs even better than the existing state-of-the-art models on the MSH WSD dataset, while our attention model was 3 or 4 times faster than our BiLSTM model with good accuracy.