What is a transformer ?

Introduction

• Date of Introduction: The concept of transformers was introduced in 2017.

• Inventor: The transformer model was invented by researchers Ashish Vaswani and his colleagues at Google Brain.

• Reference: The key paper that introduced transformers is titled "Attention is All You Need". You can read it here.

What is a transformer?

A transformer is a type of machine learning model used in artificial intelligence (AI). It is especially good at understanding and generating text, translating languages, and even recognizing patterns in data like images or music.

How does it work?
Imagine reading a sentence. Some words depend on others to make sense. For example, in "The cat chased the mouse," you know "chased" connects "cat" and "mouse." Transformers are really good at spotting these connections, no matter how far apart the words are in a sentence.

To do this, transformers use something called attention. Think of attention like a highlighter that marks which words are most important to understanding the meaning of a sentence. Unlike older methods, transformers can highlight words from anywhere in the text, not just the nearby ones.

Why is it important?

Transformers changed the game for AI because they solved a big problem: older models struggled with long texts and complex relationships between words. Transformers are fast, can process whole sentences or paragraphs at once, and capture meaning better than anything before them.

For example:

• Translating between languages became much more accurate.

• AI chatbots became much smarter and more natural.

• Tasks like summarizing articles or even creating art became possible.

What has it changed?

Transformers introduced a whole new way of thinking about AI:

1. Better Performance: Tasks like text translation and speech recognition became faster and more accurate.

2. New Models: Transformers inspired powerful AI models like GPT (used in ChatGPT) and BERT (used by search engines like Google).

3. Cross-Domain Use: While first used for text, transformers now help with images (e.g., DALL·E for art) and even protein research in biology!