Lab 3: Prompt Engineering#
Prompt engineering is the process of constructing and refining input prompts to improve the quality of generated responses from language models. It’s an iterative process that requires experimentation to find the optimal approach for a given problem.
Key strategies for effective prompt engineering include:
Writing clear and specific instructions
Highlighting or specifying the relevant parts of the prompt
Adding details or restrictions to guide the model’s output
Instructing the model to follow a step-by-step approach for complex tasks
In this lab, we’ll use the AmazonNova Micro model to demonstrate various prompt engineering techniques across different machine learning tasks.
About This Lab#
Throughout this lab, you will encounter two types of interactive elements:
No coding is needed for an activity. You try to understand a concept,
answer questions, or run a code cell.
Challenges are where you test your understanding by implementing something new or taking a short quiz.
Please work through this notebook from top to bottom to avoid errors due to missing code or context.
Table of Contents#
1. Installation of libraries#
We first install recent versions of boto3 and botocore.
!pip install --no-deps -q -r ../requirements.txt
We access the Bedrock service through boto3 by providing the service name, region name and endpoint URL.
import json, boto3
session = boto3.session.Session()
bedrock_inference = session.client(
service_name="bedrock-runtime",
region_name=session.region_name,
)
Let’s specify the API parameters. We will use the Amazon Nova Micro model.
def send_prompt(prompt_data, temperature=0.0, top_p=0.5, max_token_count=1000):
# select model
model_id = "amazon.nova-micro-v1:0"
# Build the message to call Converse API
message_content = []
message_content.append({"text": prompt_data})
messages = [
{
"role": "user",
"content": message_content,
}
]
inf_params = {"maxTokens": max_token_count, "topP": top_p, "temperature": temperature}
response = bedrock_inference.converse(
modelId=model_id, messages=messages, inferenceConfig=inf_params
)
# Process and return the response
return response["output"]["message"]["content"][0]["text"]
Example problems#
2.1 Text summarization#
With text summarization, the main purpose is to create a shorter version of a given text while preserving the relevant information in it.
We use the following text from the sustainability section of about.amazon.com.
"In 2021, we reached 85% renewable energy across our business. Our first solar projects in South Africa and the United Arab Emirates came online, and we announced new projects in Singapore, Japan, Australia, and China. Our projects in South Africa and Japan are the first corporate-backed, utility-scale solar farms in these countries. We also announced two new offshore wind projects in Europe, including our largest renewable energy project to date. As of December 2021, we had enabled more than 3.5 gigawatts of renewable energy in Europe through 80 projects, making Amazon the largest purchaser of renewable energy in Europe."
Let’s start with the first summarization example below. We pass this text as well as the instruction to summarize it. The instruction part of the prompt becomes The following is a text about Amazon. Summarize this:
prompt_data = """The following is a text about Amazon. Summarize this: \
Text: In 2021, we reached 85% renewable energy across our business.\
Our first solar projects in South Africa and the United Arab Emirates\
came online, and we announced new projects in Singapore, Japan, \
Australia, and China. Our projects in South Africa and Japan are \
the first corporate-backed, utility-scale solar farms in these \
countries. We also announced two new offshore wind projects in \
Europe, including our largest renewable energy project to date.\
As of December 2021, we had enabled more than 3.5 gigawatts of \
renewable energy in Europe through 80 projects, making Amazon \
the largest purchaser of renewable energy in Europe."""
print(send_prompt(prompt_data, temperature=0.0))
In 2021, Amazon achieved 85% renewable energy usage across its operations. The company launched its first solar projects in South Africa and the United Arab Emirates, and announced new initiatives in Singapore, Japan, Australia, and China. Notably, the solar farms in South Africa and Japan are the first corporate-backed, utility-scale solar projects in those countries. Additionally, Amazon announced two new offshore wind projects in Europe, including its largest renewable energy project to date. By December 2021, Amazon had facilitated over 3.5 gigawatts of renewable energy in Europe through 80 projects, establishing itself as the largest purchaser of renewable energy in the region.
Nice. This text is shorter. We can set the desired lenght of the summary by adding more constraints to the instructions. Let’s create a one-sentence summary of this text. The instruction part of the prompt becomes the following: The following is a text about Amazon. Summarize it in one sentence.
prompt_data = """The following is a text about Amazon. Summarize it in one sentence. \
Text: In 2021, we reached 85% renewable energy across our business.\
Our first solar projects in South Africa and the United Arab Emirates\
came online, and we announced new projects in Singapore, Japan, \
Australia, and China. Our projects in South Africa and Japan are \
the first corporate-backed, utility-scale solar farms in these \
countries. We also announced two new offshore wind projects in \
Europe, including our largest renewable energy project to date.\
As of December 2021, we had enabled more than 3.5 gigawatts of \
renewable energy in Europe through 80 projects, making Amazon \
the largest purchaser of renewable energy in Europe."""
print(send_prompt(prompt_data, temperature=0.0))
In 2021, Amazon expanded its renewable energy initiatives with new solar and offshore wind projects, becoming the largest purchaser of renewable energy in Europe and reaching 85% renewable energy usage across its business.
Nice! The model generated a one-sentence summary.
2.2 Question Answering#
In Question Answering problem, a Machine Learning model answers some questions using some provided context. Here as context, we will use the previous text about Amazon’s sustainability efforts.
The first question is asking about the names of the countries mentioned in the text.
The instruction section of the prompt is What are the names of the countries in the following text?
prompt_data = """What are the names of the countries in the following text? \
Text: In 2021, we reached 85% renewable energy across our business.\
Our first solar projects in South Africa and the United Arab Emirates\
came online, and we announced new projects in Singapore, Japan, \
Australia, and China. Our projects in South Africa and Japan are \
the first corporate-backed, utility-scale solar farms in these \
countries. We also announced two new offshore wind projects in \
Europe, including our largest renewable energy project to date.\
As of December 2021, we had enabled more than 3.5 gigawatts of \
renewable energy in Europe through 80 projects, making Amazon \
the largest purchaser of renewable energy in Europe."""
print(send_prompt(prompt_data, temperature=0.0))
The countries mentioned in the text are:
1. South Africa
2. United Arab Emirates
3. Singapore
4. Japan
5. Australia
6. China
7. Europe (Note: Europe is a continent, not a single country, but it is mentioned in the context of renewable energy projects.)
So, the specific countries are South Africa, United Arab Emirates, Singapore, Japan, Australia, and China.
Nice. We get all of the geographical places mentioned in the text.
Let’s try to learn something specific about the document. For example, the amount of gigawatts that the project in Europe enabled.
The instruction section of the prompt is How many gigawatts of energy did Amazon enable in Europe according to the following text?
prompt_data = """How many gigawatts of energy did Amazon enable in \
Europe according to the following text? \
Text: In 2021, we reached 85% renewable energy across our business.\
Our first solar projects in South Africa and the United Arab Emirates\
came online, and we announced new projects in Singapore, Japan, \
Australia, and China. Our projects in South Africa and Japan are \
the first corporate-backed, utility-scale solar farms in these \
countries. We also announced two new offshore wind projects in \
Europe, including our largest renewable energy project to date.\
As of December 2021, we had enabled more than 3.5 gigawatts of \
renewable energy in Europe through 80 projects, making Amazon \
the largest purchaser of renewable energy in Europe."""
print(send_prompt(prompt_data, temperature=0.0))
According to the provided text, Amazon enabled more than 3.5 gigawatts of renewable energy in Europe as of December 2021 through 80 projects. This achievement made Amazon the largest purchaser of renewable energy in Europe. The specific number of gigawatts enabled is "more than 3.5 gigawatts," indicating that the exact figure could be slightly higher, but the precise amount is not explicitly stated in the text.
Nice. We were able to extract that information and return in the answer.
Let’s try another example, this time without a question that doesn’t need an input text. An explicit input may not be necessary for some questions. For example, we can ask some general questions like below.
How many months are there in a year?
prompt_data = """How many months are there in a year?"""
print(send_prompt(prompt_data, temperature=0.0))
There are 12 months in a year. This is a standard measure used in the Gregorian calendar, which is the most widely used civil calendar in the world. The months, in order, are:
1. January
2. February
3. March
4. April
5. May
6. June
7. July
8. August
9. September
10. October
11. November
12. December
Each month has a specific number of days, ranging from 28 to 31, with February typically having 28 days and 29 days during a leap year.
How many meters are in a mile?
prompt_data = """How many meters are in a mile?"""
print(send_prompt(prompt_data, temperature=0.0))
There are approximately 1,609.34 meters in a mile. To be more precise, the exact conversion factor is 1,609.344 meters per mile, but for most practical purposes, rounding to 1,609.34 meters is sufficient.
What is the result when you add up 2 and 9?
prompt_data = """What is the result when you add up 2 and 9?"""
print(send_prompt(prompt_data, temperature=0.0))
When you add up the numbers 2 and 9, the result is:
\[ 2 + 9 = 11 \]
So, the sum is 11.
The answers above are correct.
2.3 Text Generation#
Text generation is one of the common use cases for Large Language Models. The main purpose is to generate some high quality text considering a given input. We will cover a few examples here.
Customer service example:
Let’s start with a customer feedback example. Assume we want to write an email to a customer who had some problems with a product that they purchased.
Write an email response from Example Corp company customer service
based on the following email that was received from a customer.
Customer email: “I am not happy with this product. I had a difficult
time setting it up correctly because the instructions do not cover all
the details. Even after the correct setup, it stopped working after
a week.”
prompt_data = """Write an email response from Example Corp company customer service \
based on the following email that was received from a customer.
Customer email: "I am not happy with this product. I had a difficult \
time setting it up correctly because the instructions do not cover all \
the details. Even after the correct setup, it stopped working after \
a week." """
print(send_prompt(prompt_data, temperature=0.0))
Subject: Re: Feedback on Your Recent Purchase
Dear [Customer's Name],
Thank you for reaching out to us and for taking the time to share your feedback regarding your recent purchase from Example Corp. We sincerely apologize to hear about your experience with the product and the issues you encountered.
We understand how frustrating it can be when a product does not meet your expectations, especially when it comes to setup and functionality. Your feedback is invaluable to us as we strive to improve our products and services continually.
To address your concerns, we would like to take the following steps:
1. **Detailed Setup Assistance**: We are committed to providing clearer and more comprehensive setup instructions. We will be revising the current documentation to ensure all necessary details are covered. In the meantime, if you need immediate assistance, please do not hesitate to contact our technical support team at [support phone number] or [support email address]. Our team is ready to help you through the setup process.
2. **Product Functionality**: We are deeply sorry to hear that the product stopped working after a week. To help resolve this issue, we would like to offer you a full technical support assessment. Please reply to this email with your order number and a brief description of the problem, and we will arrange for a technician to contact you as soon as possible.
3. **Replacement or Refund**: If the product continues to malfunction or if you would prefer a refund, we will process it promptly. You can return the product to us by following the instructions provided in the return label attached to your package or by contacting our customer service team directly.
We truly value your business and want to make things right. Please let us know if there is anything specific you would like us to address further. Your satisfaction is our top priority, and we are here to ensure you have a positive experience with Example Corp.
Thank you for your patience and understanding.
Best regards,
[Your Full Name]
Customer Service Representative
Example Corp
[Your Contact Information]
[Company Website URL]
---
If you have any immediate questions or need urgent assistance, please do not hesitate to reach out to us directly.
Nice! The generated text asks customer to provide more details to resolve the issue.
Generating product descriptions:
We can use generative AI to write creative product descriptions for our products. In the example below, we create three product descriptions for a sunglasses product.
Product: Sunglasses.
Keywords: polarized, style, comfort, UV protection.
List three variations of a detailed product
description for the product listed above, each
variation of the product description must
use at least two of the listed keywords.
prompt_data = """Product: Sunglasses. \
Keywords: polarized, style, comfort, UV protection. \
List three different product descriptions \
for the product listed above using \
at least two of the provided keywords."""
print(send_prompt(prompt_data, temperature=0.0))
Certainly! Here are three different product descriptions for the sunglasses, each utilizing at least two of the provided keywords:
### Product Description 1:
**"Experience ultimate comfort and style with our polarized sunglasses. Designed with advanced polarized lenses, these sunglasses reduce glare and enhance your vision, ensuring you stay protected from harmful UV rays. Perfect for outdoor adventures, these stylish sunglasses combine sleek design with top-notch UV protection."**
### Product Description 2:
**"Elevate your look with our stylish, UV-protected sunglasses. Featuring polarized lenses, these sunglasses not only provide superior UV protection but also offer unparalleled comfort. Whether you're at the beach or on the slopes, their chic design and polarized technology will keep your eyes safe and clear."**
### Product Description 3:
**"Our sunglasses blend comfort and cutting-edge style with exceptional UV protection. The polarized lenses reduce glare, making them ideal for any outdoor activity. Enjoy a day out in the sun with peace of mind, knowing your eyes are shielded from UV rays while looking effortlessly cool."**
2.4 In-context learning#
As pre-trained large language models learn from large and diverse data sources, they tend to build a holistic view of languages and text. This advantage allows them to learn from some input-output pairs present within the input texts.
In this section, we will explain this “in-context” learning capability with some examples. Depending on the level of information presented to the model, we can use zero-shot, one-shot or few-shot learning. We start with the most extreme case, no information presented to the model. This is called “zero-shot-learning”.
Zero-shot learning:#
Assume the model is given a translation task and an input word.
Translate English to Spanish
cat ==>
prompt_data = """Translate the following word from English to Spanish \
word: cat \
translation: """
print(send_prompt(prompt_data, temperature=0.0))
The English word "cat" translates to "gato" in Spanish.
Here is the translation:
- English: cat
- Spanish: gato
"Gato" is a masculine noun in Spanish and is used to refer to a domestic feline. It's important to note that in Spanish, nouns have genders, and "gato" is always masculine. For example, you would say "el gato" (the cat) to indicate the masculine gender.
Correctly translated to Spanish. Let’s try something different in the next one.
One-shot learning:#
We can give the model one example and let it learn from the example to solve a problem. Below, we provide an example sentence about a cat and the model completes the second sentence about a table in a similar way.
Answer the last question
question: what is a cat?
answer: cat is an animal
##
last question: what is a car?
answer: car is
prompt_data = """Answer the last question \
question: what is a cat? \
answer: cat is an animal \
## \
last question: what is a car? \
answer: car is """
print(send_prompt(prompt_data, temperature=0.0))
a vehicle. A car is a self-propelled passenger road vehicle that typically has four wheels and an internal combustion engine or an electric motor that powers it. Cars are designed to transport people and sometimes small amounts of cargo, and they come in various shapes and sizes, including sedans, hatchbacks, SUVs (sport utility vehicles), trucks, and convertibles. They are an essential mode of transportation in many parts of the world, facilitating daily commutes, travel, and various other activities. Cars are equipped with a range of features and technologies to enhance safety, comfort, and efficiency, including advanced braking systems, airbags, GPS navigation, and infotainment systems. Additionally, there is a growing trend towards electric and hybrid cars to reduce environmental impact and reliance on fossil fuels.
It worked very well.
Few-shot learning:#
We can give the model multiple examples to learn from. Providing more examples can help the model produce more accurate results. Let’s also change the style of the example answers by adding some negation to them.
Answer the last question
question: what is a car?
answer: car is not an animal
##
question: what is a cat?
answer: cat is not a vehicle
##
last question: what is a shoe?
answer: shoe is
prompt_data = """Answer the last question
question: what is a car?
answer: car is not an animal
##
question: what is a cat?
answer: cat is not a vehicle
##
last question: what is a shoe?
answer: shoe is """
print(send_prompt(prompt_data, temperature=0.0))
Answer: A shoe is not a living creature or a vehicle. It is a piece of footwear designed to protect and cover the human foot, providing comfort, support, and protection from various environmental elements. Shoes come in various styles, materials, and designs, tailored for different purposes such as casual wear, formal occasions, sports, and work. They are essential for walking, running, and other activities that involve foot movement.
The response picked up the overall style very well. See that it responded starting with “not”.
We can increase the temperature to get different responses. Let’s try that below. Try running the cell below multiple times to obtain different answers
prompt_data = """Answer the last question
question: what is a car?
answer: car is not an animal
##
question: what is a cat?
answer: cat is not a vehicle
##
last question: what is a shoe?
answer: shoe is """
print(send_prompt(prompt_data, top_p=1.0, temperature=0.85))
**Last question: What is a shoe?**
**Answer: A shoe is not an animal or a vehicle, but rather a piece of clothing designed to be worn on the feet. It typically covers the foot and sometimes part of the ankle, providing protection, support, and an element of style. Shoes come in various types and styles, including sneakers, boots, sandals, and dress shoes, each serving different purposes and fashion trends.**
Let’s try one more example. This time we remove the instruction and try to complete the last sentence.
question: what is a cat?
answer: cat is a domesticated wild animal that belongs to the Felidae family.
##
question: what is a car?
answer: car is a vehicle with wheels that is used for transportation.
##
last question: what is a shoe?
answer: shoe is
prompt_data = """
question: what is a cat?
answer: cat is a domesticated wild animal that belongs to the Felidae family.
##
question: what is a car?
answer: car is a vehicle with wheels that is used for transportation.
##
question: what is a shoe?
answer: shoe is """
print(send_prompt(prompt_data, temperature=0.0))
a foot covering designed to protect and support the human foot, typically made of leather, fabric, or other materials. Shoes come in various styles and designs, ranging from casual and athletic to formal and elegant. They are essential for providing comfort, protection, and sometimes even fashion statement. Shoes usually have a sole that provides traction and a heel or toe area that supports the wearer's weight. They are an important part of everyday attire for most people around the world.
It worked again. The model nicely followed the provided pattern.
2.5 Chain of thought concept#
Chain of thought concept breaks down a problem into a series of intermediate reasoning steps. This way of thinking has significantly improved the quality of the outputs generated by the Large Language Models.
Here is the question.
Example without Chain-of-thought (CoT)#
Answer the following question.
Question: When I was 16, my sister was half of my age.
Now, I’m 42. How old is my sister now? \
Answer:
prompt_data = """Answer the following question just with the number.
Question: When I was 16, my sister was half of my age. \
Now, I’m 42. How old is my sister now?
Answer: """
print(send_prompt(prompt_data, temperature=0.0))
36
The answer is incorrect! This is not a big surprise. Many Large Language Models make these types of mistakes. In this case, the model skipped a few steps to solve the problem.
Example with zero-shot CoT#
It is also possible to elicit reasoning from LLMs in a zero-shot situation without needing to provide one-shot examples. An explicit instruction for the model to ”think step by step“ might help the LLM find the right solution. This approach is called zero-shot chain of thought.
Answer the following question.
Question: When I was 16, my sister was half of my age.
Now, I’m 42. How old is my sister now? \
Let’s think step by step and describe all steps. \
Answer:
prompt_data = """Answer the following question.
Question: When I was 16, my sister was half of my age. \
Now, I’m 42. How old is my sister now?
Let's think step by step and describe all steps.
Answer: """
print(send_prompt(prompt_data, temperature=0.0))
To determine the current age of your sister, we need to follow these steps:
1. **Determine the age difference between you and your sister:**
- When you were 16, your sister was half your age. Therefore, your sister was \( \frac{16}{2} = 8 \) years old.
- This means there is an age difference of \( 16 - 8 = 8 \) years between you and your sister.
2. **Calculate your sister's current age:**
- You are now 42 years old.
- Since the age difference between you and your sister is always 8 years, we subtract this difference from your current age to find your sister's age.
- Your sister's current age is \( 42 - 8 = 34 \).
Therefore, your sister is currently \( \boxed{34} \) years old.
The model answers correctly! You can also choose to provide the model with examples to guide its reasoning approach. This can enable the model to adopt different and potentially more robust reasoning strategies based on the examples provided.
One-shot CoT#
Allowing the model to think step-by-step allowed the model to reason through the problem and come to the correct answer. Let’s try another idea. As we have seen in the in-context learning topic, LLMs tend to learn from the provided inputs and apply those learnings to another problems. Here, we will first provide the step-by-step solution for the problem with different numbers and then ask the model to solve the original problem. Since we are showing the model to reason by showing it one example, this approach is called one-shot chain of thought
Answer the following question:
Question: When I was 10, my sister was half of my age.
Now, I’m 70. How old is my sister now?
Answer: When I was 10 years old, my sister was half of my age.
So, the age of the sister at that time = 10/2 = 5
This implies that the sister is 5 years younger.
Now, when I’m 70 years and age of sister = 70 - 5
Age of sister = 65.
Question: When I was 16, my sister was half of my age.
Now I’m 42. How old is my sister now?
Answer:
prompt_data = """Answer the following question.
Question: When I was 10, my sister was half of my age. \
Now, I’m 70. How old is my sister now?
Answer: When I was 10 years old, my sister was half of my age. \
So, the age of the sister at that time = 10/2 = 5 \
This implies that the sister is 5 years younger. \
Now, when I’m 70 years and age of sister = 70 - 5 \
Age of sister = 65. \
Question: When I was 16, my sister was half of my age. \
Now I’m 42. How old is my sister now?
Answer: """
print(send_prompt(prompt_data, temperature=0.0))
To determine the current age of your sister, we need to follow a similar approach as in the first question.
1. When you were 16 years old, your sister was half of your age. Therefore, her age at that time was:
\[
\text{Sister's age} = \frac{16}{2} = 8 \text{ years old}
\]
2. This means your sister is 8 years younger than you. To find her current age, we subtract this age difference from your current age:
\[
\text{Your current age} = 42 \text{ years}
\]
\[
\text{Sister's current age} = 42 - 8 = 34 \text{ years}
\]
Therefore, your sister is currently \( \boxed{34} \) years old.
The model followed the given example and applied the same steps to solve the problem.
3. Quiz Questions#
Well done on completing the lab! Now, it’s time for a brief knowledge assessment.
Try it Yourself!#
Answer the following questions to test your understanding of basic prompt engineering practices.
import sys
sys.path.append('..')
from mlu_utils.quiz_questions import *
lab3_question1.display()
lab3_question2.display()