Photo by Courtney Cook on Unsplash
Read More
  • 61 views
  • 13 minute read

Chain-of-Thought, CoT

The performance of LLMs on reasoning tasks has undergone substantial change in recent years with the introduction of Chain-of-Thought (CoT) prompting. This technique guides an LLM to produce step-by-step intermediate reasoning, enabling the model to exhibit a human-like structure of thought. As task complexity increases, however, the limitations of traditional CoT have become more apparent, motivating a series of follow-up methods designed to address these issues. This article presents an overview of CoT and its extensions.
Read More
0
0
0
0
Photo by Tomáš Malík on Unsplash
Read More
  • 82 views
  • 11 minute read

Vision Transformer Model

In the field of image recognition, Convolutional Neural Networks (CNNs) have long been the dominant architecture. In recent years, Transformer models have achieved great success in Natural Language Processing (NLP), which has led researchers to consider applying the Transformer architecture to image processing tasks. Vision Transformer (ViT) is a model designed for image understanding based on the Transformer framework.
Read More
0
0
0
0
Photo by Sestrjevitovschii Ina on Unsplash
Read More
  • 85 views
  • 4 minute read

Layer Normalization

Normalization is a data transformation technique originating from statistics. It adjusts the mean and variance of data to make it more stable and predictable. In deep learning, normalization is widely used to improve the stability and efficiency of model training. This article explains the original concept of normalization, introduces the design and limitations of batch normalization, and explores how layer normalization addresses these issues to become a standard component in modern language models.
Read More
0
0
0
0
Photo by Koushik Chowdavarapu on Unsplash
Read More
  • 134 views
  • 6 minute read

Adam Optimizer

When training neural networks, choosing a good optimizer is critically important. Adam is one of the most commonly used optimizers, so that it has almost become the default choice. Adam is built upon the foundations of SGD, Momentum, and RMSprop. By revisiting the evolution of these methods, we can better understand the principles behind Adam.
Read More
0
0
0
0
Photo by jean wimmerlin on Unsplash
Read More
  • 133 views
  • 10 minute read

LoRA: Low-Rank Adaptation of Large Language Models

When LLMs often have tens of billions of parameters, performing a single fine-tuning run can exhaust an entire GPU. LoRA (Low-Rank Adaptation of Large Language Models) offers a clever solution: instead of modifying the model’s original parameters directly, it learns new knowledge through low-rank matrices. This allows us to adapt the model’s behavior quickly and at very low cost, while still preserving its original performance.
Read More
0
0
0
0
Photo by Farhan Khan on Unsplash
Read More
  • 87 views
  • 5 minute read

CLIP Model

CLIP (Contrastive Language-Image Pre-training) is a model proposed by OpenAI in 2021. It achieves strong generalization capability by integrating visual and language representations, and it has extensive potential applications. This article will introduce both the theory and practical implementation of CLIP.
Read More
0
0
0
0
Photo by Logan Armstrong on Unsplash
Read More
  • 325 views
  • 15 minute read

Generative Pre-trained Transformer, GPT

Over the past decade in the field of Natural Language Processing (NLP), the Generative Pre-trained Transformer (GPT) has undoubtedly been one of the most iconic technologies. GPT has not only redefined the approach to language modeling but also sparked a revolution centered around pre-training, leading to the rise of general-purpose language models. This article begins with an overview of the GPT architecture and delves into the design principles and technological evolution from GPT-1 to GPT-3.
Read More
0
0
0
0