Surendhar Reddy

AI/LLM notes


Overview of LLMs and GPT

A beginner-friendly guide to generative language models

GPT language model

Eight Things to Know about Large Language Models

The widespread public deployment of large language models (LLMs) in recent months has prompted a wave of new attention and engagement from advocates, policymakers, and scholars from many fields. This attention is a timely response to the many urgent questions that this technology raises, but it can sometimes miss important considerations. This paper surveys the evidence for eight potentially surprising such points:

  1. LLMs predictably get more capable with increasing investment, even without targeted innovation.
  2. Many important LLM behaviors emerge unpredictably as a byproduct of increasing investment.
  3. LLMs often appear to learn and use representations of the outside world.
  4. There are no reliable techniques for steering the behavior of LLMs.
  5. Experts are not yet able to interpret the inner workings of LLMs.
  6. Human performance on a task isn’t an upper bound on LLM performance.
  7. LLMs need not express the values of their creators nor the values encoded in web text.
  8. Brief interactions with LLMs are often misleading.

Overview of Neural Networks and Machine Learning

Neural Network, Machine Learning and Weights


(vs convolutional neural networks)

Attentional Neural Network Models

Attention fundamentals + intuition of attention

Other reading material

Prompt engineering

What is prompting

Prompts as programming

Finetuning GPT

Effective prompt programming

GPT’s weakness

Blind Prompting

Some tactical advice on prompting

Other examples + use cases

Customising GPT

Word vector embeddings

Running an LLM locally or building one from scratch

Running locally (Yes, it’s possible)

Building your own LLM

Reinforcement Learning from Human Feedback (RLHF)

Minqi Jiang explains RLHF

RLHF is a technique that fine-tunes language models by introducing a reward signal based on human preference data. This creates a bias in the model towards generating completions that are favored by humans, improving the reliability of the answers but reducing the diversity of generations that can be sampled. The choice of humans for collecting the preference data is important as the model will exhibit their values. Reinforcement learning is mode-seeking, introducing biases into the model, and can be used to prune out bad modes in the distribution of text on the internet. However, this can also lead to loss of diversity in exchange for more reliable generations.


The AI tech stack

Building LLM applications

Prompt versioning and optimization

Most papers on prompt engineering are tricks that can be explained in a few sentences

The cost of LLMOps is in inference

Latency in LLMOps

The impossibility of cost + latency analysis for LLMs

Prompting vs. finetuning vs. alternative

Emeddings and vector databases

Backward and forward compatibility

Most common enterprise use case for LLMs

The most popular enterprise application is to talk to the knowledge base / data. Many, many startups are building tools to let enterprise users query their internal data and policies in natural languages or in the Q&A fashion. Some focus on verticals such as legal contracts, resumes, financial data, or customer support. Given a company’s all documentations, policies, and FAQs, you can build a chatbot that can respond your customer support requests.

The main way to do this application usually involves these 4 steps:

  1. Organize your internal data into a database (SQL database, graph database, embedding/vector database, or just text database)
  2. Given an input in natural language, convert it into the query language of the internal database. For example, if it’s a SQL or graph database, this process can return a SQL query. If it’s embedding database, it’s might be an ANN (approximate nearest neighbor) retrieval query. If it’s just purely text, this process can extract a search query.
  3. Execute the query in the database to obtain the query result.
  4. Translate this query result into natural language.

Generative interfaces beyond chat

Good dialogue interfaces should:

  1. Have agents that coinhabit your workspace
  2. Take full advantage of rich shared context
  3. Lead with constrained “happy path” actions
  4. Fall back to chat as an escape hatch
  5. Speed up iteration for fast feedback loops

State of GPT

Notes from talk by Andrej Karpathy at Mircosoft Build