Text Generation: Model Training

10 minute read



This is the first post in a series on generating text with recurrent neural networks (RNNs). We will be using the R interface to Keras for training and making predictions with our models. All of the code to reproduce this analysis has been echoed and is also available on my Github.

First, we load several R packages necessary for our analysis:

library("tidyverse"); theme_set(theme_bw()); theme_update(panel.grid.minor = element_blank())


A (Brief) Introduction to Neural Networks

Neural networks are powerful models which are capable of expressing highly non-linear relationships. This is accomplished via the composition of linear functions and non-linear activation functions. Typically, observations of \(p\) variables are taken as input, mapped through “hidden layers,” and ultimately transformed to predict some response. This is illustrated in the graphic below, reproduced from Wikipedia: