Improving Text with ChatGPT: Top_P, Frequency Penalty, Presence Penalty

Table of Contents
graphic of a laptop with information representing accessibility

Recently, I’ve been toying with ChatGPT, the brainchild of OpenAI. You’re wondering why? Well, I’m a bit of a technomad, a web developer, and an SEO wizard with a decade-long track record. My job, my passion, is to whip up quality content that stands out in the endless sea of web pages

In my latest escapades with ChatGPT, I’ve stumbled upon the sacred art of parameter tuning, the real MVP when it comes to crafting high-caliber text. It’s like adjusting the knobs on a cosmic radio to tune into the right channel. I found that when you set Top P to a solid 0.5, and give a gentle nudge to Frequency Penalty and Presence Penalty, say around 0.7, you’ve hit the sweet spot. I’m almost always on temp 0.1 because consistency is key, but it depends on the app; it’s just as much an art as a science

This potent combo gives rise to text that is as diverse as a tropical rainforest and as creative as a jazz improv, but without spiraling into utter nonsense. It’s the golden mean, my friend, where relevance shakes hands with coherence

In this devlog, I’ll share my experience with ChatGPT and these specific parameters, and provide tips and best practices for other developers and content creators looking to improve their text generation capabilities. Let’s dive in!

Section 1: Top P

In this section, we’ll examine the impact of the top_p parameter in GPT-3 on the output text diversity, while keeping the temperature constant at 0.5. To demonstrate this, we’ve generated text samples using three different top_p values: 0, 0.5, and 1. In the table below, you can see the generated text samples

Top_PSample 1Sample 2Sample 3
0The perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedroomsThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedroomsThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to stay in a beachfront villa with a private pool and direct access to the beach. The villa would have all the amenities I need, including a fully-equipped kitchen, comfortable bedrooms
0.5The perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green jungles. I would love to spend my days exploring the island, swimming in the ocean, and relaxing in the sun. I would also love to go snorkeling and scuba diving to discover the vibrant underwater lifeThe perfect vacation destination for me would be a place with a mix of culture, nature, and adventure. I would love to explore a city with a rich history, vibrant culture, and amazing architecture. I would also want to be able to take part in outdoor activities such as hiking, biking, and kayaking. I would also like to haveThe perfect vacation destination for me would be a tropical island with white sand beaches, crystal clear blue waters, and lush green forests. I would love to stay in a luxurious beachfront villa with a private pool and breathtaking views of the ocean. During the day, I would explore the island, snorkel in the warm waters, and lounge
1The perfect vacation destination for me would be a secluded beach on a tropical island. The beach would be surrounded by lush green palm trees and crystal clear turquoise waters. The sand would be soft and white and the sun would be shining. I would have my own private beach hut to stay in, with a comfortable bed and a hammockThe perfect vacation destination would be somewhere with a mix of outdoor activities, cultural experiences, and relaxation. It should have beautiful beaches and plenty of sunshine, with plenty of hiking trails and other outdoor activities to explore. There should be a vibrant nightlife and a variety of restaurants and bars to enjoy. The city should have plenty of cultural attractions to exploreThe perfect vacation destination for me would be a tropical island with crystal clear blue waters, white sandy beaches, and lush green jungles. The island should have plenty of activities to do, like snorkeling, scuba diving, kayaking, and hiking. There should be plenty of restaurants and bars to explore, as well as plenty of

Explanation

Stepping into the pulsating core of our technomad journey, we’re gonna unwrap the juicy secrets behind how our trusty sidekick, ChatGPT, churns out its magic. When you’ve got the top_p tuned to a cold, rigid zero, our buddy ChatGPT goes full auto-pilot mode, spewing out identical outputs across all samples. It’s like a monotonous DJ, dishing out the most probable beats at every step, leading to a repetitive playlist lacking diversity

Bump up that top_p to a breezy 0.5, and voila! Our AI buddy starts to mix things up. Each output’s a unique groove, painting a distinct, vacation-worthy picture. It’s like a jukebox that’s had a sip of creativity juice, pulling from a broader palette of tokens, conjuring up a diverse cocktail of outputs

Crank that top_p all the way up to a blazing 1, and our ChatGPT friend goes full ‘freestyle’, sampling from its entire token smorgasbord. The resulting outputs are a wild mix, but hey, they’re still coherent and on point, only more varied than the staid outputs at lower top_p values. Though, keep an eye out when you max out those tokens; you don’t want the narrative ship to veer off course!

Top_p, my friend, is like your AI DJ’s creative dial. Lower values churn out more focused, predictable outputs, while cranking it up brings in a medley of more varied, creative tunes. Need to whip up some artsy, out-of-the-box content? Turn that top_p up to a cool 0.5 or 1, and let ChatGPT unfurl its imaginative wings. But if you’re gunning for a professional tone, keep that top_p low, maybe at a sensible 0.3, or even closer to 0, ensuring you get focused, coherent, and dependable outputs

Why is it called Top_P

Top_P has got its name as it points to the proportion, or the ‘P’ of the highest likelihood tokens considered during the spellcasting, err, text generation process. The ‘Top’ part signifies that ChatGPT is picking out the top probability tokens based on their likelihood of being the next token in the sequence, and the ‘P’ denotes the slice of these tokens involved in the process

In short, Top P 1.0 is like saying to ChatGPT, “Go on, use all tokens in the vocabulary.”

Section 2: Frequency Penalty

While tinkering with ChatGPT, I’ve also played around with the Frequency Penalty parameter. By tweaking this value, you can control how much of a repeat offender your generated text is going to be. Let’s dig into this, shall we?

To demonstrate this, let’s examine the results of a simple prompt:

Model:  gpt-3.5-turbo-0613
Prompt: Product Description for a cotton t-shirt

with a temperature of 0, max-tokens at 256 and Top P set to 0, while adjusting the Frequency Penalty

Frequency PenaltyExampleWord Frequencies
0Introducing our premium cotton t-shirt, the perfect blend of comfort, style, and durability. Crafted with the finest quality cotton fabric, this t-shirt is designed to provide a soft and breathable feel against your skincotton:2, tshirt:2, the:2, and:2, introducing:1, our:1, premium:1, perfect:1, blend:1, of:1, comfort:1, style:1, durability:1, crafted:1, with:1, finest:1, quality:1, fabric:1, this:1, is:1, designed:1, to:1, provide:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
1Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1, 100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1
2Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1, 100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1

Vocabulary variation and frequency penalties, oh my! The difference between a frequency penalty of 2 and 1, in this case, is about as significant as a molehill next to a mountain – it’s not. With just two extra unique words popping up in the higher penalty example, and a smidgen of repetition in both, it’s clear that a game-changer it is not

Now, let’s add a frequency penalty of 0 into the mix. What do we get? Fewer unique words (36, to be exact) compared to its higher-penalty cousins, and slightly less repetition, with only four repeated words. You’d expect a dramatic difference, but the impact of frequency penalty on vocabulary variation, in this context, is as compelling as a potato at a fruit party

Section 3: Presence Penalty

So, we’ve dealt with frequency penalties. Now let’s wade into the waters of presence penalties

Frequency PenaltyExampleWord Frequencies
0Introducing our premium cotton t-shirt, the perfect blend of comfort, style, and durability. Crafted with the finest quality cotton fabric, this t-shirt is designed to provide a soft and breathable feel against your skincotton:2, tshirt:2, the:2, and:2, introducing:1, our:1, premium:1, perfect:1, blend:1, of:1, comfort:1, style:1, durability:1, crafted:1, with:1, finest:1, quality:1, fabric:1, this:1, is:1, designed:1, to:1, provide:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
1Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric, ensuring a soft and breathable feel against your skincotton:2, tshirt:2, and:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1:100:1, pure:1, fabric:1, ensuring:1, a:1, soft:1, breathable:1, feel:1, against:1, your:1, skin:1
2Introducing our premium cotton t-shirt, the perfect blend of comfort and style. Crafted with utmost care, this t-shirt is made from 100% pure cotton fabric that feels incredibly soft against your skincotton:2, tshirt:2, introducing:1, our:1, premium:1, the:1, perfect:1, blend:1, of:1, comfort:1, and:1, style:1, crafted:1, with:1, utmost:1, care:1, this:1, is:1, made:1, from:1:100:1, pure:1, fabric:1, that:1, feels:1, incredibly:1, soft:1, against:1, your:1, skin:1

Like the yin to frequency penalty’s yang, the presence penalty controls the likelihood of selecting tokens that have already made their debut in the generated text. Frequency penalty, on the other hand, plays gatekeeper to high-frequency tokens during text generation

It looks like it has more effect on function words like ‘the, and’:

Presence Penalty at zero
Presence Penalty = 0
Presence Penalty at one
Presence Penalty = 1
Presence Penalty at two
Presence Penalty = 2

Section 4: Examples of Text Generation with ChatGPT

Roll up your sleeves, my friend, because we’re about to showcase the capabilities of our trusty sidekick, ChatGPT, complete with different combinations of Top P, Frequency Penalty, and Presence Penalty:

Top PFrequency PenaltyPresence PenaltyGenerated TextCommentary
0.00.00.0Practicing yoga can have many benefits for the mind and body, including increased flexibility, improved strength and balance, reduced stress and anxiety, and better overall health and well-beingThis text is informative and covers many benefits of yoga, but may be too generic and lack a personal touch
0.50.00.0Yoga can provide many benefits for the mind and body, including improved flexibility, strength, and relaxation. It can also reduce stress and anxiety, and improve overall well-beingThis text is concise and informative, with a good balance of different benefits of yoga. It would be suitable for a general audience
0.00.50.0Practicing yoga regularly can help you improve your physical and mental health. It can increase flexibility, reduce stress and anxiety, and promote relaxation and mindfulnessThis text is personal and engaging, with a focus on the individual benefits of yoga. It would be suitable for a more targeted audience, such as those interested in personal growth or mindfulness
0.00.00.5Yoga is a powerful practice that can benefit the mind and body in many ways. By practicing yoga, you can improve your physical and mental health, as well as reduce stress and anxietyThis text is concise and informative, with a focus on the overall benefits of yoga. It would be suitable for a general audience
0.50.50.0Yoga is a great way to improve your physical and mental health. It can increase flexibility and strength, reduce stress and anxiety, and promote overall well-beingThis text is engaging and personal, with a focus on the physical and mental benefits of yoga. It would be suitable for a more targeted audience, such as those interested in fitness or stress reduction
0.50.00.5Practicing yoga can provide a variety of benefits for the mind and body, including increased flexibility, reduced stress and anxiety, and improved physical and mental healthThis text is informative and engaging, with a focus on the different benefits of yoga. It would be suitable for a more general audience, such as those interested in fitness or well-being
0.00.50.5Yoga is a holistic practice that can benefit both the mind and body. It can improve flexibility, reduce stress and anxiety, and promote overall well-beingThis text is informative and personal, with a focus on the holistic benefits of yoga. It would be suitable for a more targeted audience, such as those interested in mindfulness or self-care
0.50.50.5Yoga is an effective way to improve your physical and mental health. It can increase flexibility, reduce stress and anxiety, and promote relaxation and mindfulnessThis text is concise and engaging, with a focus on the different benefits of yoga. It would be suitable for a general audience, such as those interested in fitness or stress reduction

As you can see, ChatGPT is not some monotonous machine, spewing out the same gibberish. No siree! Tweak its parameters, and you’ll see a world of difference in the quality and diversity of its outputs. It’s like a jukebox with infinite songs. With the right mix of Top P, Frequency Penalty, and Presence Penalty, you can make ChatGPT hum to your specific tune

But remember, this ain’t no substitute for good ol’ human-generated content. It’s a tool, a sidekick, to help you create. You’re the boss. Your judgement and expertise are what will ensure the content is accurate, enlightening, and useful for your audience

How do we use this information? The secret’s in the prompts

tl;dr: Visit this page to learn more about prompt examples.

Ever found yourself in awe of an eloquent dance of words, spun by the ever-entertaining ChatGPT? Wondering what the trick is? Let me tell you, it’s all in the prompts. Those invaluable linguistic keys we use to unlock the expressive power of our AI companion

Sure, the dials and knobs like Top P and fpr Frequency Penalty ChatGPT come into play. But the real showstopper, the golden ticket to the heart of the system, is the prompt engineering. We’re about to scratch the surface of an unexplored universe. Fasten your seatbelts; this exploration will be anything but shallow

Consider ‘Chain of Thought’ prompts for a moment. They’re like the pied pipers, leading ChatGPT into a harmonious exchange, allowing it to weave threads of logic from previous statements and maintain a seamless dialogue. Then you’ve got the ‘Tree Thinking’ prompts, the architects of thought. They advocate a systematic, tree-branching approach to problem-solving

‘Step-by-Step’ prompts, they’re the tour guides. Ideal for navigating ChatGPT through labyrinthine tasks, laying out every step of the process, rather than just showcasing the finale. ‘Compression Prompts’, on the other hand, are the shrink rays, condensing elaborate directives into concise, token-friendly nuggets

Commands such as “Err on the side of too much information”, they’re the whistleblowers. They prompt ChatGPT to spill all its beans on a topic. Meanwhile, “then do X” prompts are the choreographers. They sequence ChatGPT’s actions as if it were rehearsing for a well-coordinated performance

There’s a whole spectrum of prompts at play – scenario prompts, character prompts, point-of-view prompts. Each shapes ChatGPT, molding its responses to different situations, personifying it, or enabling it to adopt diverse perspectives. We even have prompts that foster common sense, creativity, decisiveness, specificity, and a plethora of other skills in our digital companion. It’s like a buffet of cognitive delights!

Mastering advanced prompting is like learning the language of ChatGPT. The prompts are your magical incantations, your secret ingredient to summon the perfect prose as per your whim. While parameters provide a great polish, the true value is squeezed out by the perfect prompts

Still got an itch for more? Ready to dive headfirst into the sea of advanced prompting techniques? We’ve got your back! Head over to prompt examples and immerse yourself in a pool of wisdom. Let’s elevate your ChatGPT prowess to unseen heights!

tl;dr:

ChatGPT is one helluva weapon in your text generation arsenal. By mastering techniques like Top P, Frequency Penalty, and Presence Penalty, you control the unpredictability and diversity of generated text, avoiding repetition, and mundane content

Here’s a summary of the benefits of each technique:

  • Top_P: The maestro of creativity and coherence, it lets you dictate how much the model stays on script or goes off on a tangent
  • Frequency Penalty: The muse encouraging the model to find diverse ways of expressing an idea, upping the overall quality and variety of the output
  • Presence Penalty: The bouncer that keeps a check on overuse or repetition of specific words or phrases, coaxing the model to spin more varied and engaging responses

What is the frequency penalty in Chatgpt?

Frequency penalty is a parameter that allows you to control the repetition of words or phrases in ChatGPT’s responses. By increasing the frequency penalty value (e.g., 0.5 or above), you discourage the model from generating the same words multiple times, resulting in a more varied and engaging output⁴.

What is the presence penalty in OpenAI?

Presence penalty is a parameter used to encourage the model to include a diverse range of tokens in the generated text. It is a value that is subtracted from the log-probability of a token each time it is generated. The presence penalty reduces a word’s probability if it has previously been used in the projected text⁵.

What is the presence penalty in GPT-3?

Presence penalty in GPT-3 works the same way as in OpenAI, as GPT-3 is one of the language models offered by OpenAI. You can use presence penalty to influence the overall quality and diversity of the generated text by adjusting its value²⁶.

What is the difference between frequency and presence penalty?

The difference between frequency and presence penalty is that frequency penalty works by adding a value to the log-probability of a token each time it occurs in the generated text, while presence penalty works by subtracting a value from the log-probability of a token each time it is generated. Frequency penalty discourages repeated tokens, while presence penalty encourages diverse tokens⁵.

Source: Conversation with Bing, 2023-07-14

  1. Mastering ChatGPT Prompts: A Guide to Using Parameters
  2. Difference between frequency and presence penalties?
  3. The Ultimate Guide to OpenAI’s GPT-3 Language Model – Twilio
  4. OpenAI Platform
  5. Frequency penalties – Dreamsavior
  6. Gpt-3.5-turbo and frequency/presence penalty – API

See more posts like this!

Nolan Phelps
Nolan Phelps

nolan@funkpd.com

With over a decade of experience in web development and a passion for SEO, I bring a wealth of real-life skills and constant research to the table. I've tackled projects for international hotels, mining corporations, and equipment manufacturers, and I specialize in website performance and speed optimization like no other. I'm here to empower you with my expertise and tips for website success through my blog.

All Posts
Interested? Curious? Bored? Let's start a conversation!
funkproductions