EbookQA
ProcessIntermediate

How is the Epicurious Recipes dataset prepared for use in LSTM text generation according to the chapter?

The Epicurious Recipes dataset is prepared by loading the data, filtering it to include only recipes with a title and description, and then converting the text into a TensorFlow Dataset. The text is vectorized using a Keras TextVectorization layer, which converts text to lowercase, assigns integer tokens to the most common 10,000 words, and pads or clips sequences to a length of 201 tokens. The dataset is then used to create a training set by shifting the tokenized sequences to generate input-output pairs for the LSTM model.

The preparation of the Epicurious Recipes dataset for LSTM text generation involves several steps. First, the dataset is loaded and filtered to retain only those recipes that have both a title and a description. The text data is then transformed into a TensorFlow Dataset and processed using a Keras TextVectorization layer. This layer standardizes the text to lowercase, tokenizes the most frequent 10,000 words, and adjusts the sequence length to 201 tokens by padding or clipping as necessary. The vectorized text is then used to create a training dataset, where each input sequence is paired with a target sequence that is shifted by one token. This setup allows the LSTM model to learn to predict the next word in a sequence.

Key points

  • Load and filter the dataset to include only recipes with a title and description.
  • Convert the text data into a TensorFlow Dataset.
  • Use a Keras TextVectorization layer to process the text: lowercase, tokenize, and pad/clip sequences.
  • Create a training set by shifting tokenized sequences to form input-output pairs for the LSTM model.
Source:Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play· Autoregressive Models· p. 160–176

Related questions

Cover of Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play

Generative Deep Learning: Teaching Machines to Paint, Write, Compose, and Play

David Foster;

Second Edition · O’Reilly Media, Inc.

View this ebook