Improving Text with ChatGPT: Top_P, Frequency Penalty, Presence Penalty

Top Posts

A detailed image of a word cloud composed of various 'Lorem Ipsum' words. The words should be arranged in a visually striking pattern that forms the silhouette of an open book. The most commonly used Lorem Ipsum words should be more prominent in size. The color scheme should be a gradient ranging from classic book-page cream to a digital blue, symbolizing the transition from traditional text to digital text. The background should be a simple, dark color to make the word cloud stand out.
A futuristic AI-themed workspace with multiple holographic screens displaying various parameters and code such as GPT presence penalty

Recently, I’ve been toying with ChatGPT, the brainchild of OpenAI. You’re wondering why? Well, I’m a bit of a technomad, a web developer, and an SEO wizard with a decade-long track record. My job, my passion, is to whip up quality content that stands out in the endless sea of web pages.

TL;DR?

Watch this blog here if you’re short on time.

In my latest escapades with ChatGPT, I’ve stumbled upon the sacred art of parameter tuning, the real MVP when it comes to crafting high-caliber text. It’s like adjusting the knobs on a cosmic radio to tune into the right channel. I found that when you set Top P to a solid 0.5, and give a gentle nudge to Frequency Penalty and Presence Penalty, say around 0.7, you’ve hit the sweet spot. I’m almost always on temp 0.1 because consistency is key, but it depends on the app; it’s just as much an art as a science

This potent combo gives rise to text that is as diverse as a tropical rainforest and as creative as a jazz improv, but without spiraling into utter nonsense. It’s the golden mean, my friend, where relevance shakes hands with coherence

A futuristic AI-themed workspace with multiple holographic screens displaying various parameters and code, hinting at text generation

In this devlog, I’ll share my experience with ChatGPT and these specific parameters, and provide tips and best practices for other developers and content creators looking to improve their text generation capabilities. Let’s dive in!

Section 1: Top P

In this section, we’ll examine the impact of the top_p parameter in GPT-3 on the output text diversity, while keeping the temperature constant at 0.5. To demonstrate this, we’ve generated text samples using three different top_p values: 0, 0.5, and 1. In the table below, you can see the generated text samples

Top_PSample 1Sample 2Sample 3
0The perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedroomsThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedroomsThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedrooms
0.5The perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to spend my days exploring the island, swimming in the ocean, and relaxing in the sun. I would also love to go snorkeling and scuba diving to discover the vibrant underwater lifeThe perfect vacation destination for me would be a place with a mix of culture, nature, and adventure. I would love to explore a city with a rich history, vibrant culture, and amazing architecture. I would also want to be able to take part in outdoor activities such as hiking, biking, and kayaking. I would also like to haveThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green forests. I would love to stay in a luxurious beachfront villa with a private pool and breathtaking views of the ocean. During the day, I would explore the island, snorkel in the warm waters, and lounge
1The perfect vacation destination for me would be a secluded beach on a tropical island. The beach would be surrounded by lush green palm trees and crystal clear turquoise waters. The sand would be soft and white and the sun would be shining. I would have my own private beach hut to stay in, with a comfortable bed and a hammockThe perfect vacation destination would be somewhere with a mix of outdoor activities, cultural experiences, and relaxation. It should have beautiful beaches and plenty of sunshine, with plenty of hiking trails and other outdoor activities to explore. There should be a vibrant nightlife and a variety of restaurants and bars to enjoy. The city should have plenty of cultural attractions to exploreThe perfect vacation destination for me would be a tropical island with crystal clear blue waters, white sandy beaches, and lush green jungles. The island should have plenty of activities to do, like snorkeling, scuba diving, kayaking, and hiking. There should be plenty of restaurants and bars to explore, as well as plenty of

Explanation

Stepping into the pulsating core of our technomad journey, we’re gonna unwrap the juicy secrets behind how our trusty sidekick, ChatGPT, churns out its magic. When you’ve got the top_p tuned to a cold, rigid zero, our buddy ChatGPT goes full auto-pilot mode, spewing out identical outputs across all samples. It’s like a monotonous DJ, dishing out the most probable beats at every step, leading to a repetitive playlist lacking diversity

Bump up that top_p to a breezy 0.5, and voila! Our AI buddy starts to mix things up. Each output’s a unique groove, painting a distinct, vacation-worthy picture. It’s like a jukebox that’s had a sip of creativity juice, pulling from a broader palette of tokens, conjuring up a diverse cocktail of outputs

Crank that top_p all the way up to a blazing 1, and our ChatGPT friend goes full ‘freestyle’, sampling from its entire token smorgasbord. The resulting outputs are a wild mix, but hey, they’re still coherent and on point, only more varied than the staid outputs at lower top_p values. Though, keep an eye out when you max out those tokens; you don’t want the narrative ship to veer off course!

An AI DJ with Black descent, surrounded by a futuristic DJ booth with advanced equipment and holographic interfaces.

Top_p, my friend, is like your AI DJ’s creative dial. Lower values churn out more focused, predictable outputs, while cranking it up brings in a medley of more varied, creative tunes. Need to whip up some artsy, out-of-the-box content? Turn that top_p up to a cool 0.5 or 1, and let ChatGPT unfurl its imaginative wings. But if you’re gunning for a professional tone, keep that top_p low, maybe at a sensible 0.3, or even closer to 0, ensuring you get focused, coherent, and dependable outputs.

ELI5: Top_P

Got it, let’s simplify it:

When ChatGPT talks, it picks from a huge pile of words. Top_P is like a rule that tells ChatGPT how many words it can look at before it picks one to use.

Here’s how it works, step by step, like you’re picking a treat from a huge bag:

  1. You ask ChatGPT something. This is like reaching into the bag for a treat.
  2. ChatGPT looks at the treats. Instead of looking at all the treats in the bag, Top_P tells it to only look at the best ones at the top.
  3. Top_P decides how many treats to look at.
  • If Top_P is big: ChatGPT can look at lots of treats and pick a really special one.
  • If Top_P is small: ChatGPT only looks at a few treats and picks one of the usual ones.

So, Top_P helps ChatGPT decide if it’s going to say something usual or something more surprising and creative. It’s like choosing between grabbing the first treat you touch or looking for the best treat in the bag.

Why is it called Top_P

Top_P has got its name as it points to the proportion, or the ‘P’ of the highest likelihood tokens considered during the spellcasting, err, text generation process. The ‘Top’ part signifies that ChatGPT is picking out the top probability tokens based on their likelihood of being the next token in the sequence, and the ‘P’ denotes the slice of these tokens involved in the process

In short, Top P 1.0 is like saying to ChatGPT, “Go on, use all tokens in the vocabulary.”

Section 2: Frequency Penalty

While tinkering with ChatGPT, I’ve also played around with the Frequency Penalty parameter. By tweaking this value, you can control how much of a repeat offender your generated text is going to be. Let’s dig into this, shall we?

To demonstrate this, let’s examine the results of a simple prompt:

Model:  gpt-3.5-turbo-0613
Prompt: Product Description for a cotton t-shirt

with a temperature of 0, max-tokens at 256 and Top P set to 0, while adjusting the Frequency Penalty

Frequency PenaltyExampleWord Frequencies
0Introducing our premium cotton t-shirt, the perfect blend of comfort, style, and durability. Crafted with the finest quality cotton fabric, this t-shirt is designed to provide a soft and breathable feel against your skincotton:2, tshirt:2, the:2, and:2, introducing:1, our:1, premium:1, perfect:1, blend:1, of:1, comfort:1, style:1, durability:1, crafted:1, with:1, finest:1, quality:1, fabric:1, this:1, is:1, designed:1, to:1, provide:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
1Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1, 100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1
2Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1, 100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1

Vocabulary variation and frequency penalties, oh my! The difference between a frequency penalty of 2 and 1, in this case, is about as significant as a molehill next to a mountain – it’s not. With just two extra unique words popping up in the higher penalty example, and a smidgen of repetition in both, it’s clear that a game-changer it is not.

Now, let’s add a frequency penalty of 0 into the mix. What do we get? Fewer unique words (36, to be exact) compared to its higher-penalty cousins, and slightly less repetition, with only four repeated words. You’d expect a dramatic difference, but the impact of frequency penalty on vocabulary variation, in this context, is as compelling as a potato at a fruit party

ELI5: Frequency Penalty

The Frequency Penalty is a setting used when ChatGPT is creating its responses. Imagine ChatGPT has a deck of cards, where every card has a word on it. When it’s ChatGPT’s turn to speak, it plays a card to add a word to the conversation.

still not getting it

Absolutely, let’s dive into the concept of Frequency Penalty with a straightforward approach:

  1. Imagine ChatGPT as a kid in a candy store, where the candies are words.
  2. Frequency Penalty is like a little reminder for the kid.
  • Without the reminder: The kid might grab the same candy (word) too many times because it’s their favorite.
  • With the reminder: The kid remembers to try different candies (words), not just their favorite one over and over.
  1. How the Frequency Penalty reminder works:
  • Each candy (word) starts with a full ‘yumminess’ score. The kid wants the yummiest candies.
  • Each time the kid takes a candy (uses a word), that candy becomes a little less yummy (less desirable).
  • Candies (words) that become less yummy get picked less. The kid looks for other yummy candies instead.
  1. Adjusting the reminder changes the kid’s choices:
  • A strong reminder (high Frequency Penalty): The kid is really encouraged to try all different candies, not just the favorites. After taking one, it quickly becomes less appealing.
  • A gentle reminder (low Frequency Penalty): The kid can still take their favorite candies, but they’re just a little less yummy each time, so they might still come back to them sometimes.
  1. Why use the Frequency Penalty?
  • To keep the conversation interesting. Just like trying different candies makes the visit to the store more fun, using a variety of words makes ChatGPT’s responses more engaging.
  • To avoid sounding repetitive. It helps ChatGPT avoid saying the same thing the same way too many times, which can get boring.

The Frequency Penalty is like a subtle nudge to keep exploring new words and not just stick to what’s been said before, making sure every ‘candy’ gets a chance to shine in the conversation.

Section 3: Presence Penalty

So, we’ve dealt with frequency penalties. Now let’s wade into the waters of presence penalties

Frequency PenaltyExampleWord Frequencies
0Introducing our premium cotton t-shirt, the perfect blend of comfort, style, and durability. Crafted with the finest quality cotton fabric, this t-shirt is designed to provide a soft and breathable feel against your skincotton:2, tshirt:2, the:2, and:2, introducing:1, our:1, premium:1, perfect:1, blend:1, of:1, comfort:1, style:1, durability:1, crafted:1, with:1, finest:1, quality:1, fabric:1, this:1, is:1, designed:1, to:1, provide:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
1Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric, ensuring a soft and breathable feel against your skincotton:2, tshirt:2, and:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1:100:1, pure:1, fabric:1, ensuring:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
2Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1:100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1

Like the yin to frequency penalty’s yang, the presence penalty controls the likelihood of selecting tokens that have already made their debut in the generated text. Frequency penalty, on the other hand, plays gatekeeper to high-frequency tokens during text generation

It looks like it has more effect on function words like ‘the, and’:

Presence Penalty at zero
Presence Penalty = 0
Presence Penalty at one
Presence Penalty = 1
Presence Penalty at two
Presence Penalty = 2

ELI5: Presence Penalty

Okay, think of it like when you’re playing a game of tag. In this game, once you’ve been “it,” you get a little tired each time you’re tagged again. If you’re really tired, you’re less likely to be “it” again because you’re running slower. The Presence Penalty in ChatGPT works a bit like that. It’s a rule that makes the AI less likely to use the same words over and over again. So, if a word has been “it” a lot, the AI tries to tag a new word next time, keeping the conversation full of different words.

still not getting it

Okay, let’s break it down super simple, step by step:

  1. ChatGPT is like a toy box full of words. It can reach in and pick any word to use when it talks to you.
  2. The Presence Penalty is like a rule for picking words.
  • Without the rule: ChatGPT might keep picking the same favorite words over and over, which can be boring.
  • With the rule: ChatGPT tries to pick different words it hasn’t used much yet, to make the conversation more interesting.
  1. How the Presence Penalty rule works:
  • Every word has points. When ChatGPT talks, it uses words with the most points.
  • If a word is used, it loses points. The more it’s used, the fewer points it has.
  • Words with fewer points get less chance to be picked. So, ChatGPT looks for other words with more points – words it hasn’t used as much.
  1. What happens when we change the rule:
  • If we make the rule strict (high Presence Penalty): ChatGPT avoids repeating words even more. It’s like making words lose a lot of points each time they are used.
  • If we make the rule lenient (low Presence Penalty): ChatGPT can use the same words more often. It’s like words lose only a few points, so they can still be picked again soon.
  1. Why use the Presence Penalty?
  • To make ChatGPT sound more natural. Like how people try to use different words to explain things, so they don’t sound repetitive.
  • To keep the conversation fresh. It encourages ChatGPT to think of new ways to say something, which can be more fun and interesting.

So, the Presence Penalty helps ChatGPT to not sound like a broken record, using the same words again and again. It encourages variety and keeps the chat lively!

Section 4: Examples of Text Generation with ChatGPT

Roll up your sleeves, my friend, because we’re about to showcase the capabilities of our trusty sidekick, ChatGPT, complete with different combinations of Top P, Frequency Penalty, and Presence Penalty:

Top PFrequency PenaltyPresence PenaltyGenerated TextCommentary
0.00.00.0Practicing yoga can have many benefits for the mind and body, including increased flexibility, improved strength and balance, reduced stress and anxiety, and better overall health and well-beingThis text is informative and covers many benefits of yoga, but may be too generic and lack a personal touch
0.50.00.0Yoga can provide many benefits for the mind and body, including improved flexibility, strength, and relaxation. It can also reduce stress and anxiety, and improve overall well-beingThis text is concise and informative, with a good balance of different benefits of yoga. It would be suitable for a general audience
0.00.50.0Practicing yoga regularly can help you improve your physical and mental health. It can increase flexibility, reduce stress and anxiety, and promote relaxation and mindfulnessThis text is personal and engaging, with a focus on the individual benefits of yoga. It would be suitable for a more targeted audience, such as those interested in personal growth or mindfulness
0.00.00.5Yoga is a powerful practice that can benefit the mind and body in many ways. By practicing yoga, you can improve your physical and mental health, as well as reduce stress and anxietyThis text is concise and informative, with a focus on the overall benefits of yoga. It would be suitable for a general audience
0.50.50.0Yoga is a great way to improve your physical and mental health. It can increase flexibility and strength, reduce stress and anxiety, and promote overall well-beingThis text is engaging and personal, with a focus on the physical and mental benefits of yoga. It would be suitable for a more targeted audience, such as those interested in fitness or stress reduction
0.50.00.5Practicing yoga can provide a variety of benefits for the mind and body, including increased flexibility, reduced stress and anxiety, and improved physical and mental healthThis text is informative and engaging, with a focus on the different benefits of yoga. It would be suitable for a more general audience, such as those interested in fitness or well-being
0.00.50.5Yoga is a holistic practice that can benefit both the mind and body. It can improve flexibility, reduce stress and anxiety, and promote overall well-beingThis text is informative and personal, with a focus on the holistic benefits of yoga. It would be suitable for a more targeted audience, such as those interested in mindfulness or self-care
0.50.50.5Yoga is an effective way to improve your physical and mental health. It can increase flexibility, reduce stress and anxiety, and promote relaxation and mindfulnessThis text is concise and engaging, with a focus on the different benefits of yoga. It would be suitable for a general audience, such as those interested in fitness or stress reduction

As you can see, ChatGPT is not some monotonous machine, spewing out the same gibberish. No siree! Tweak its parameters, and you’ll see a world of difference in the quality and diversity of its outputs. It’s like a jukebox with infinite songs. With the right mix of Top P, Frequency Penalty, and Presence Penalty, you can make ChatGPT hum to your specific tune

But remember, this ain’t no substitute for good ol’ human-generated content. It’s a tool, a sidekick, to help you create. You’re the boss. Your judgement and expertise are what will ensure the content is accurate, enlightening, and useful for your audience

How do we use this information? The secret’s in the prompts

tl;dr: Visit this page to learn more about prompt examples.

Ever found yourself in awe of an eloquent dance of words, spun by the ever-entertaining ChatGPT? Wondering what the trick is? Let me tell you, it’s all in the prompts. Those invaluable linguistic keys we use to unlock the expressive power of our AI companion

Sure, the dials and knobs like Top P and fpr Frequency Penalty ChatGPT come into play. But the real showstopper, the golden ticket to the heart of the system, is the prompt engineering. We’re about to scratch the surface of an unexplored universe. Fasten your seatbelts; this exploration will be anything but shallow

Use ChatGPT? Want to learn how to make an online business with ChatGPT, work while traveling, and network with other nomads?

Bridging Realities: The Art of Networking for the 21st Century Technomad - Nolan Phelps

Consider ‘Chain of Thought’ prompts for a moment. They’re like the pied pipers, leading ChatGPT into a harmonious exchange, allowing it to weave threads of logic from previous statements and maintain a seamless dialogue. Then you’ve got the ‘Tree Thinking’ prompts, the architects of thought. They advocate a systematic, tree-branching approach to problem-solving

‘Step-by-Step’ prompts, they’re the tour guides. Ideal for navigating ChatGPT through labyrinthine tasks, laying out every step of the process, rather than just showcasing the finale. ‘Compression Prompts’, on the other hand, are the shrink rays, condensing elaborate directives into concise, token-friendly nuggets.

A person with South Asian descent, depicted as a futuristic explorer, is navigating a complex, digital terrain with the aid of 'Step-by-Step' holographic prompts.

Commands such as “Err on the side of too much information”, they’re the whistleblowers. They prompt ChatGPT to spill all its beans on a topic. Meanwhile, “then do X” prompts are the choreographers. They sequence ChatGPT’s actions as if it were rehearsing for a well-coordinated performance

There’s a whole spectrum of prompts at play – scenario prompts, character prompts, point-of-view prompts. Each shapes ChatGPT, molding its responses to different situations, personifying it, or enabling it to adopt diverse perspectives. We even have prompts that foster common sense, creativity, decisiveness, specificity, and a plethora of other skills in our digital companion. It’s like a buffet of cognitive delights!

Mastering advanced prompting is like learning the language of ChatGPT. The prompts are your magical incantations, your secret ingredient to summon the perfect prose as per your whim. While parameters provide a great polish, the true value is squeezed out by the perfect prompts

Still got an itch for more? Ready to dive headfirst into the sea of advanced prompting techniques? We’ve got your back! Head over to prompt examples and immerse yourself in a pool of wisdom. Let’s elevate your ChatGPT prowess to unseen heights!

tl;dr:

ChatGPT is one helluva weapon in your text generation arsenal. By mastering techniques like Top P, Frequency Penalty, and Presence Penalty, you control the unpredictability and diversity of generated text, avoiding repetition, and mundane content

Here’s a summary of the benefits of each technique:

  • Top_P: The maestro of creativity and coherence, it lets you dictate how much the model stays on script or goes off on a tangent
  • Frequency Penalty: The muse encouraging the model to find diverse ways of expressing an idea, upping the overall quality and variety of the output
  • Presence Penalty: The bouncer that keeps a check on overuse or repetition of specific words or phrases, coaxing the model to spin more varied and engaging responses

Want to learn about Custom GPT’s? We’ve done some research here!

What is the frequency penalty in Chatgpt?

Frequency penalty is a parameter that allows you to control the repetition of words or phrases in ChatGPT’s responses. By increasing the frequency penalty value (e.g., 0.5 or above), you discourage the model from generating the same words multiple times, resulting in a more varied and engaging output⁴.

What is the presence penalty in OpenAI?

Presence penalty is a parameter used to encourage the model to include a diverse range of tokens in the generated text. It is a value that is subtracted from the log-probability of a token each time it is generated. The presence penalty reduces a word’s probability if it has previously been used in the projected text⁵.u003cbru003e

What is the presence penalty in GPT-3?

Presence penalty in GPT-3 works the same way as in OpenAI, as GPT-3 is one of the language models offered by OpenAI. You can use presence penalty to influence the overall quality and diversity of the generated text by adjusting its value²⁶.u003cbru003e

What is the difference between frequency and presence penalty?

The difference between frequency and presence penalty is that frequency penalty works by adding a value to the log-probability of a token each time it occurs in the generated text, while presence penalty works by subtracting a value from the log-probability of a token each time it is generated. Frequency penalty discourages repeated tokens, while presence penalty encourages diverse tokens⁵.u003cbru003e

Source: Conversation with Bing, 2023-07-14

  1. Mastering ChatGPT Prompts: A Guide to Using Parameters
  2. Difference between frequency and presence penalties?
  3. The Ultimate Guide to OpenAI’s GPT-3 Language Model – Twilio
  4. OpenAI Platform
  5. Frequency penalties – Dreamsavior
  6. Gpt-3.5-turbo and frequency/presence penalty – API

General Knowledge Quiz

See more posts like this!

Was this content helpful?

Interested? Curious? Bored? Let's start a conversation!
funkproductions