How GPT-3 works on the examples.

Andrei Ivanouski
3 min readDec 13, 2020

--

Generative Pre-trained Transformer 3 (GPT-3) is an autoregressive language model that uses deep learning to produce human-like text. It is the third-generation language prediction model in the GPT-n series created by OpenAI, a San Francisco-based artificial intelligence research laboratory.

Your phone prediction neural network predicts the next word when you write your message.

next word prediction

What is the Next Word Prediction?
The easiest way to write the sentence is to type the first word, and then select the second word by using predictive text. It is fast and very easy. It is also a great feature for mobile users but there is one problem when using this solution. The predictive text only works for the next word, it will not help you with the second word.

How GPT-3 Works on next word prediction?
The Graph Predictive Text uses a neuron network to predict the next word. This is done by using the connections between the words. The Graph Predictive Text has several layers.
Graph Predictive Text has several layers of connections:
The first layer is the Input Layer.
This layer has all the words of the dictionary. The input layer has connections with the next layer, it is called the Hidden Layer.
The next layers are the Hidden Layers.
Their layers are used to predict the next word by using the words in the previous layer.
The next layer is the Output Layer.

Neural network example

This layer is used to predict the next word by using the words in the input layer.

Now we are ready to predict the next word.
The first step is to check the input word. The Input word is checked in the Input Layer to find the input word. The Input word is checked in the Hidden Layer to find the next word. The next word is checked in the Output Layer to find the next word.
The second step is to calculate the score for each word. To calculate the score the Graph Predictive Text calculates the score for each word and the best word is selected.
The third step is to predict the best word. The best word is selected from the output layer.
Now we are ready to predict the next word.

GPT-2 visualized example

Text above was generated by AI © inite.io

Sign up to discover human stories that deepen your understanding of the world.

Free

Distraction-free reading. No ads.

Organize your knowledge with lists and highlights.

Tell your story. Find your audience.

Membership

Read member-only stories

Support writers you read most

Earn money for your writing

Listen to audio narrations

Read offline with the Medium app

--

--

No responses yet

Write a response