type
status
date
slug
summary
tags
category
icon
password
Treat LLMs as inexperienced assistants who need detailed job explanations.
Background
With the explosive popularity of ChatGPT 3.5, large language models (LLMs) have become the industry’s darling over the past year or two. Major tech giants have released their own trained LLMs, so every now and then, a new large model emerges, claiming to outperform the current reigning champion. As a humble data worker, it’s neither feasible nor within my capabilities to train my own LLM. Instead, my focus is on continuously updating my understanding of various models (a never-ending task) and figuring out how to get LLMs to complete the tasks I assign—in other words, prompt engineering.
Recently, I’ve had to handle related tasks at work, so during the holidays, I watched Professor Hung-yi Lee’s lecture recordings from National Taiwan University (links at the end). This helped me systematically review some prompt-writing techniques and realize that I’ve already been using some of them at work—haha. This post summarizes the more universally applicable prompt-writing notes I’ve compiled.
Technique 1: Magic Phrases
Phrases like "Let’s think about it step by step" or "Answer by starting with analysis" are tested and proven "magic phrases" in academia that effectively enhance LLM performance. You can even try having the LLM generate its own magic phrases (though humans might not always understand them). However, these phrases may not work equally well on every model. For instance, the latest ultra-large parameter models can achieve similar performance without needing such prompts.
Technique 2: Provide More Context
As mentioned earlier, we can treat the LLM like a new assistant who isn’t very skilled yet and explain the task to it. Providing more information—such as task prerequisites, additional context, or examples (in-context learning)—helps the model better understand what to do. However, research shows that not all LLMs can interpret examples effectively, so it’s best to provide examples only when using the latest ultra-large parameter models.
Technique 3: Self-Reflection (Self-Consistency)
Think back to when we took exams in school—after finishing, we’d always check our answers to ensure we hadn’t made mistakes. This technique is like prompting the LLM to review its response, verify its correctness, and only then submit it. After all, calculating the correct answer is hard, but verifying whether an answer is wrong is easier. Interestingly, Claude’s design incorporates a similar approach, with a hidden layer that checks whether the LLM’s responses comply with ethical guidelines, ensuring harmless outputs.
Technique 4: Break Down Tasks
This technique resembles the human approach of chain-of-thought reasoning or designing a data science solution pipeline—breaking a large task into smaller steps. Then, prepare individual prompts for each step in sequence, and even incorporate the self-reflection technique mentioned earlier. (This is exactly the solution I proposed at work, though I only realized it in hindsight—haha.)
Summary
The above techniques are all methods that everyday users can apply, even when using LLMs on platforms like POE. These skills are more than sufficient for small to medium-scale tasks. There are also more advanced approaches currently in vogue, such as Retrieval-Augmented Generation (RAG) and multi-agent systems, but I’ll save those for a future post.
Appendix: Links to Professor Hung-yi Lee’s Lecture Recordings
[Introduction to Generative AI 2024] Lectures 3–5: Can’t Train AI? Train Yourself Instead
(Part 1) — Magic Phrases and Providing More Context
