Buyer Persona Creation with AI Part 2: Should you use public LLMs for Persona Creation?

Comparing AI-generated buyer personas

After reading the previous section, you might realize that creating personas with the internal data science team is quite complex. Data science capabilities are extremely scarce, and many marketers have the hope that using publicly available LLMs such as Chat GPT can do the job with a simple prompt. But again, these are Language Models rather than Knowledge Models, and they surely will sound good, but validating them becomes crucial since these models hallucinate. In the following section, we will compare the outputs of well-known LLMS to insights our system generates to highlight sections that require close attention. Consider that these outputs are sorely trained on public data. Therefore, valuable insights based on internal data can not be considered.

If you prefer watching, here is the video:

The prompt that was inserted in various public LLMS was the following:

“Create a complete buyer persona profile for the most significant customer segment of Salesforce Marketing Cloud users. The persona profiles should contain the following sections: Demographics, a bio, a section on the personas' needs, pain points, and emotions, which Salesforce Marketing Cloud features are most important for that persona, interests, personality, communication strategies, and ad types. Also, give me a percentage of the size of this customer segment.”

Here is the ChatGPT output: Sarah Marketing Pro

Demographics

• Name: Sarah Marketing Pro
• Age: 30-45
• Gender: Female
• Location: Urban and Suburban areas
• Occupation: Marketing Manager or Director
• Company Size: Medium to Large Enterprises
• Industry: E-commerce, Technology, Finance

Bio
Sarah is an accomplished marketing professional with a Bachelor's degree in Marketing and over a decade of experience in the industry. She is passionate about leveraging technology to enhance marketing strategies and currently leads a team in a dynamic and competitive industry.

Needs
Sarah seeks a comprehensive marketing solution that seamlessly integrates and automates her multi-channel marketing efforts. She needs a platform that enables targeted and personalized communication with her audience, streamlines campaign management and provides robust analytics for data-driven decision-making.

Pain Points

• Fragmented data and disparate marketing tools
• Inefficient campaign management
• Lack of personalization in communication
• Difficulty in tracking and measuring ROI
• Limited scalability of current marketing efforts

Emotions
Sarah is eager to find a solution that not only simplifies her daily tasks but also empowers her team to deliver impactful and measurable results. She is frustrated with the current disjointed tools and is motivated to adopt a platform that can elevate the effectiveness of her marketing campaigns.

Most Essential Features of the Product
For a persona like Sarah Marketing Pro, who is a seasoned marketing professional looking for a comprehensive solution, several features of Salesforce Marketing Cloud are particularly important:

1. Journey Builder

Importance: Enables the creation of personalized, multi-channel customer journeys.
Benefit to Sarah: Allows for automation of marketing campaigns across various channels, ensuring a consistent and targeted customer experience.

2. Email Studio
Importance: Provides tools for designing, automating, and analyzing email campaigns.
Benefit to Sarah: Allows for creating personalized and engaging email content, as well as tracking the performance of email campaigns.

3. Audience Studio
Importance: Offers advanced segmentation and targeting capabilities.
Benefit to Sarah: Helps create highly targeted and personalized campaigns by segmenting the audience based on various criteria.

4. Social Studio
Importance: Facilitates social media marketing and listening.
Benefit to Sarah: Enables integrating social media into her overall marketing strategy, helping engage with the audience and monitor social conversations.

Additional requested features the LLM spit out were:

• Advertising Studio
• Analytics Builder
• Integration with CRM
• Mobile Studio
• Personalization and Customization Tools
• Scalability

Interests

• Staying updated on the latest marketing trends and technologies
• Attending industry conferences and webinars
• Networking with other marketing professionals
• Continuous learning and skill development

Personality
Sarah is a tech-savvy, ambitious, and results-oriented individual. She values innovation and is open to exploring new strategies to stay ahead in the competitive marketing landscape. She is a collaborative leader who encourages creativity and strategic thinking within her team.

Communication Strategies
Sarah prefers clear, concise, and data-driven communication. She appreciates personalized interactions highlighting the benefits and impact of her marketing efforts. Webinars, case studies, and live demos effectively engage her in decision-making.

Ad Types
Sarah responds well to informative and visually appealing content. Video ads, infographics, and success stories showcasing real-world implementations will likely capture her attention. She values authenticity and evidence of tangible results.

Segment Size
This buyer persona represents approximately 30% of Salesforce Marketing Cloud users. This segment comprises mid to large-sized enterprises with a dedicated marketing team that leverages technology to achieve their business goals.

So what are your first thoughts after reading this profile?

Sounds all pretty good, doesn't it? And exactly this is the problem- it all sounds good, but is it also true?

To validate the output, we generated additional prompts, researched the Salesforce customer base, and trained proprietary models specifically on that case to compare the outputs.

And I would say the results speak for themselves. Let us start with the segment size. Of course, understanding the importance and size of each customer segment is the absolute basis for allocating resources.

Accordingly, Chat GPT was asked to define all additional customer segments sizes of Salesforce Marketing Cloud and here is the output:

• Segment 1:Sarah Marketing Pro: 30%
• Segment 2: Alex Tech Innovator: 25%
• Segment 3: SME Marketer: 20%
• Segment 4: Enterprise Marketing Executive: 25%
• Segment 5: E-commerce Specialist: 15%
• Segment 6: Content Marketing Strategist: 10%
• Segment 7: Healthcare Marketer: 8%
• Segment 8: Financial Services Marketer: 7%
• Segment 9: Education Industry Marketer: 5%
• Segment 10: Nonprofit Marketing Coordinator: 5%

Now we do some math and look at the total percentage and we get, yes you calculated correctly, it equals to 150% total market representation.

In addition, when you compare the actual numbers, you see major differences. For instance, the system spits out 8% of healthcare marketers. However, the actual customer segment size in healthcare is 3,1%.

But we can take this even further by looking at the actual demographics of the users. Based on a report by Start. Io 72% of the Salesforce users are male. In reverse, you can ask yourself if it makes sense that the largest customer segment based on ChatGPT is female with a 30% representation- probably not. Accordingly, if you take such information for granted, you might start marketing campaigns that are based on completely false assumptions.

Another example is the ad type preferences. Particular AI models can identify ad preferences based on their personalities. There is a full article on customer psychographics and ad preferences, so if you are interested, check it out. In our case, we asked ChatGPT to clarify how it defined the preferences, and here is the answer:

“I must clarify that the information provided in the persona profiles, including preferences for ad types, is a generalization based on common trends and preferences observed in marketing practices. It's not specific knowledge about an individual named Sarah or any particular user.”

Frankly, when I read the word generalization, my toes cramp since personas, and all marketing endeavors nowadays aim to move away from generalization towards personalization. So how helpful is that information?

We asked the same validation question for the personality part, and the answer was:

“The information provided in the persona profiles, including personality traits, is a generalized representation based on common characteristics often associated with individuals in certain roles or industries. It is not derived from specific data about.”

Sounds a bit suspicious, no? Well, we asked which model ChatGPT uses to define personality traits, and here comes the answer:

“I don't use specific psychological models or frameworks like the Big Five Personality Traits, Myers-Briggs Type Indicator (MBTI), or others. My responses are generated based on patterns and information present in the data on which I was trained. I don't possess the capability to conduct psychological assessments or interpret personality traits like a specialized tool or expert would.”

The translation is that the system spits out an answer based on generic scraped information and, therefore highly susceptible to bias. Oh, and since we talk about personality, whenever someone in your company raves about Myers-Briggs, just let them read the Wikipedia article. Let us say personas generated with public LLMs and Myer Briggs have one thing in common - they are arbitrary.

The last part we should highlight here is the most essential product features. This section is the most important for many marketers since the provided information guides what to mention when selling the product to that particular customer segment. The problem here is that the output was a list of standard features, and when you ask for features for the other segments, they tend to be all the same. Personalization is nowhere around.

And it gets worse - after reentering the same prompt in ChatGPT again, it spits out different demographics and required product features - hallucination at its best.

To compare, here is the output using Google BART LLM based on the same prompt:

Buyer Persona Profile: Marketing-Savvy Entrepreneur

• Demographics
• Age: 35-55 years old
• Gender: Male and female
• Education: Bachelor's degree or higher
• Occupation: Owner or CEO of a small to medium-sized business (SMB)
• Industry: Various, including retail, healthcare, technology, and services
• Location: Global, with a strong presence in North America and Europe

Bio
Sarah Jones is the co-founder and CEO of a successful digital marketing agency. She is passionate about leveraging technology to connect with customers and drive business growth. Sarah always seeks new tools and strategies to improve her marketing campaigns. She is also an active member of the marketing community and enjoys attending conferences and workshops.

Needs
Sarah needs a marketing automation platform that can help her scale her business and reach more customers. She is looking for a solution that is easy to use, scalable, and affordable. She also wants a platform that offers a variety of marketing automation features, such as email marketing, social media marketing, and lead nurturing.

Pain Points
Lack of time: She juggles multiple responsibilities and doesn't have time to manage her marketing campaigns manually.
Ineffective marketing campaigns: Her current marketing campaigns are not generating the results she needs.
Inconsistent brand messaging: Her marketing efforts are inconsistent with her brand message.

Emotions
Ambitious: She always strives to grow her business and reach her goals.
Empowered: She believes in the power of marketing to drive business growth.
Frustrated: She is frustrated with her current marketing efforts and is looking for a solution to improve them.

Salesforce Marketing Cloud Features
Email marketing: She wants to be able to send targeted email campaigns that nurture leads and convert them into customers.
Social media marketing: She wants to be able to use social media to engage with her audience and build relationships.
Lead nurturing: She wants to be able to nurture leads from the moment they interact with her business until they are ready to buy.

Interests

• Marketing automation
• Digital marketing trends
• Business growth strategies
• Marketing technology

Personality
Sarah is a results-oriented and data-driven leader. She is also a creative and innovative thinker. She is always looking for new ways to improve her marketing campaigns.

Communication Strategies
The most effective communication strategies for reaching Sarah include:
Email: Sarah is an active email user and is likely to open and read emails from people she knows and trusts, such as marketing professionals.
Social media: Sarah is active on social media and is engaged with various marketing and business communities.
Content marketing: Sarah is a voracious reader who always seeks new information and insights.

Ad Types
Display ads: Sarah is likely to see display ads on websites and social media platforms she visits.
Content marketing: Sarah is likely to engage with content marketing that is relevant to her interests and industry.
Native advertising: Sarah is likely to be receptive to native advertising that appears in a format similar to the content she is already consuming.

Percentage of Customer Segment
The Marketing-Savvy Entrepreneur customer segment makes up approximately 40% of Salesforce Marketing Cloud users. This segment is highly valuable to Salesforce because it comprises businesses looking for a powerful and scalable marketing automation solution.

What are your first initial thoughts? Better or worse? Well, at least the systems chose the name - Sarah. We can also dissect the different components as we did before with the same output. All are very generic and sometimes simply false.

Let us take again the demographic for instance - “Male and female between the ages of 35 to 55 male and female and owner or CEO of a small to medium-sized business.” Actual demographics of Salesforce display that 53% belong to large corporations with more than 10k employees, and 32% belong to companies with 1-10k employees. How do those numbers match? Well, they don't.

We then asked for the remaining customer segment sizes and the output was the following:

• Digital Marketing Professional: 30%
• Marketing-Focused Executive: 25%
• Tech-Savvy Marketer: 15%
• Marketing-Driven Startup: 10%

So the math game again and yep, you will reach 120% of the total market representation. I suggest math classes for both models at this point. As a general rule based on over 400 persona projects - usually a customer segment market representation varies between 10 - 25%.

When you think about it, it totally makes sense. 40% representation in one persona? 40% of all customers have the same pain points, requested features, or personalities? Highly doubtful I would say.

Don't get me wrong, LLMs are fantastic and make marketers' lives much easier, however, for purposes like these I highly recommend a certain amount of skepticism when using them.

The same applies to certain external vendors that use AI to create buyer personas. Some vendors offer buyer persona creation SaaS solutions for a few dollars. To be honest, something is off here. Generating data-driven personas requires a vast amount of computation time that costs money and there is no way to analyze millions of data points for a few dollars. And of course, the results speak for themselves. According to the tool that generates personas based on Google Analytics and public data, the biggest customer cluster is female, 32 years old, and works at a marketing agency- once again far away from reality. There is a full article on using external buyer persona creation tools to create, which will shed more light on the different options.

Conclusion

As we've seen, public LLMs can generate buyer personas that sound convincing at first glance-but a closer look quickly reveals inaccuracies, contradictions, and overgeneralizations that can lead to misguided marketing strategies. While these tools are powerful for ideation, they are not built to uncover real behavioral truths or validate insights with grounded data. That distinction is critical when the goal is to align strategy, product, and messaging with actual customer needs rather than convenient narratives.

In our next blog post, we'll take this discussion a step further by showcasing a buyer persona built using our proprietary models. This will allow you to see the difference between generic, surface-level profiles and data-driven personas that reveal the why behind customer behavior-insights you can actually act on.