The Unreasonable Effectiveness of the Transformer Spell Checker

Posted on Thu 18 June 2020 in Spell Checker • Tagged with Transformer, fairseq

xfspell — the Transformer Spell Checker

NOTE: All the code and pre-trained model necessary for running this spell checker can be found in the xfspell repository.

In the modern world, spell checkers are everywhere. Chances are your web browser is equipped with a spell checker which tells you when you make …


Continue reading

Training a Shakespeare Reciting Monkey using RL and SeqGAN

Posted on Thu 02 May 2019 in Text Generation • Tagged with Text Generation, RL, SeqGAN, AllenNLP

Tackle the Monkey First

There is this article titled Tackle the monkey first written by Astro Teller, who leads Google's moonshot lab "X." The article begins "Let’s say you’re trying to teach a monkey how to recite Shakespeare while on a pedestal. How should you allocate your time …


Continue reading

A Gentle Introduction to Skip-gram (word2vec) Model — AllenNLP ver.

Posted on Sat 02 February 2019 in Word Embeddings • Tagged with Word Embeddings, word2vec, AllenNLP

The Skip-gram model (so called "word2vec") is one of the most important concepts in modern NLP, yet many people simply use its implementation and/or pre-trained embeddings, and few people fully understand how the model is actually built. In this article, I'll cover:

  • What the Skip-gram model is
  • How to …

Continue reading

How to Convert an AllenNLP model and Deploy on Caffe2 and TensorFlow

Posted on Wed 09 January 2019 in Part-of-speech Tagging • Tagged with Part-of-speech Tagging, Sequential Labeling, AllenNLP, ONNX, Caffe2, TensorFlow

In the last three posts, I talked mainly about how to train NLP models using AllenNLP. AllenNLP is a great NLP framework especially for quickly prototyping research models. However, it is not optimized for scalability or portability, and it comes with a ton of dependencies that are handy when you …


Continue reading

Building Seq2Seq Machine Translation Models using AllenNLP

Posted on Thu 06 December 2018 in Machine Translation • Tagged with Machine Translation, Seq2Seq, AllenNLP

In the past two posts, I introduced how to build a sentiment analyzer using AllenNLP and how to improve it using ELMo. AllenNLP is a very powerful framework that enables you to train many other NLP models with little to no code. In this post, I'll explain how to train …


Continue reading

Improving a Sentiment Analyzer using ELMo — Word Embeddings on Steroids

Posted on Sat 27 October 2018 in Sentiment Analysis • Tagged with Sentiment Analysis, Word Embeddings, ELMo, AllenNLP

In the previous post, I showed how to train a sentiment classifier from the Stanford Sentiment TreeBank. Thanks to a very powerful deep NLP framework, AllenNLP, we were able to write the entire training pipeline in less than 100 lines of Python code.

In this post, I'm going to explain …


Continue reading

Training a Sentiment Analyzer using AllenNLP (in less than 100 lines of Python code)

Posted on Sat 13 October 2018 in Sentiment Analysis • Tagged with Sentiment Analysis, AllenNLP

What is Sentiment Analysis?

Sentiment analysis is a popular text analytic technique used in the automatic identification and categorization of subjective information within text. The technique is widely used in quantifying opinions, emotions, etc. that are usually written in an unstructured way; and thus, hard to quantify otherwise. Sentiment analysis …


Continue reading