Content Oyster - logo
Common mistakes with AI-generated UX content

4 common mistakes with AI-generated UX content—and how to avoid them

There’s no doubt about whether or not AI (artificial intelligence) is here or causing disruption. It’s everywhere. And everyone (or a lot of folks) have been asking the same question since ChatGPT’s release in 2022: “is it going to take our jobs?” Well, it looks like things are here to stay—especially when it comes to writing.

This might be a surprising stance on AI from a UX content agency, but we do see the benefits of using it. ChatGPT, particularly ChatGPT 4, creates some decent outputs if you’re specific and detailed with your prompt and do quality control. Keyword being decent here.

Where we start to have questions is when we see that AI-generated UX content is the only type of content on a website, platform, or app, and there isn’t a UX writer or content designer included in the process.

Who’s doing content quality analysis? How does the team know when criteria is met if there isn’t a writer involved?

"AI will bring you a good first draft. Only humans can turn these drafts into world-class products."

AI can be super helpful when you’re getting a product or feature kicked off. It beats the use of lorem ipsum, that’s for sure. But at the end of the day, a human who writes is still the best way to communicate with and understand other humans, and should be working with AI tools or collaborating with others doing so.

The rise of AI-generated UX content

Teams have been leaning into AI tools more and more since ChatGPT exploded onto the scene in 2022, and we can’t blame them. Tech is a fast-paced environment after all, and wanting to be first and fail fast come with the territory.

When AI is used correctly, it’s a great tool for productivity and efficiency and helps teams save time, making the repeatable and duplicable easy. The same rings true for AI for UX writing and content design:

  • It can help with brainstorming and ideation
  • It can help create or refine a string of text
  • It can help create iterations or alternatives for that string of text
  • It can help structure information

And so much more.

We’d like to think this is how all teams and their plentiful UX content humans are working together, but some dreams remain dreams. From speaking and working with different teams and clients, we’ve learned that there are some gaps between these expectations and the reality on ground, and we want to share some ways to get around them.

Common mistakes with AI-generated UX content

Mistake #1: Relying on AI a lot with little human oversight

What this might look like:

  • You’re using AI for content in your designs because you don’t have enough writers
  • You just launched a new product or feature and want to test it quickly or scale fast

Why this is less than ideal: When you use AI to craft your content, it doesn’t have your team’s user research, customer stories, or legacy knowledge as context, which happen to be the foundation of user-centered content. For this reason, AI writing outputs tend to lack personalization and can feel generic or even repetitive. You’ve probably seen your fair share of “In conclusion”, “leverage”, or “elevate”; we have.

Solution: Empower your team to use AI as a writing tool rather than a writer’s replacement. Instead of generating all of your content with AI, you can use it for brainstorming and coming up with ideas. If you do use AI for a lot of your product content, then book some time with your UX writer or content designer to audit the content and overall flow with you before it’s delivered. They’ll be able to identify content improvements and nuances that AI may have missed.

💡 Hint: If it takes more than 30 minutes for you and your content person to go through a flow with AI-generated content, they probably need to be more involved than they are. Ask your UX writer or content designer if they’re available for continued syncs (or work asynchronously) to get your content right.

Mistake #2: Misalignment with the team’s content standards

What this might look like:

  • Generating content without a style guide or content design system as reference
  • Not using voice or tone prompts with writing generators

Why this is less than ideal: With very specific prompts, AI can generate decent outputs, or at least something good enough to tame writer’s block and stimulate creativity. Without those details though, you’re at the mercy of whatever AI gives you—it’s not always good, and it certainly isn’t always accurate.

With time, this can lead to alienation from your brand’s identity, especially if older content was done by writers working with internal content guidelines that helped them maintain consistent messaging throughout the product.

Solution: When we talk about AI and content, we often say that it needs to be done correctly. That means very specific prompts by the people on your team who know your brand’s writing and language best—your writers in most cases.

💡 Hint: If AI is going to be a part of your team’s UX writing or content design process (it should be), make sure you have content quality procedures in place. This can include reviewing all AI-generated UX content for readability, accuracy, originality, and adherence with UX content governance.

Mistake #3: Underestimating user experience

What this might look like:

  • AI-generated error messages that are unclear or don’t have solutions for stuck users
  • Generated content that is irrelevant and doesn’t meet the user where they are in the flow (Example: a user seeing text for canceling a service when they just want to delete an item)

Why this is less than ideal: Content that’s irrelevant to your user can leave them feeling confused or frustrated, especially if it’s during an error state. In general though, if AI-generated content misses the mark with your users, this can lead to decreased engagement and poor user feedback for your product or feature.

Solution: Take an iterative approach with your AI-generated content and test it with actual humans. Your team’s UX writer or UX researcher can take the lead here and get content in front of your customers with a number of tests. This can include A/B testing, user interviews, or card sorting to name a few.

💡 Hint: Do your best to go into testing with an open mind and recognize any biases that may come up. Work with your research team to help you think through ideas and make sure the test you’re doing is relevant for what you want to solve.

Mistake #4: Neglecting UX content strategy

What this might look like:

  • You’re using ChatGPT in place of a UX content human. 🥲

This is a common scenario in the startup scene. The product’s launched, the company’s funded and scaling, and there’s a team of UX people, but none of them are writers. We’ve heard various ends of the spectrum from, “we haven’t gotten around to hiring one yet” to “we use AI for that”.

Why this is less than ideal: While this might work in the early stages of your product or feature, we don’t recommend it long term because it isn’t sustainable. At some point, your users will tell you that their journey is confusing with friction points because AI-generated content isn’t getting the job done. This could be user drop-offs due to unusable or annoying experiences, or an increased volume of customer complaints because of ineffective content.

Solution: AI tools are just that—tools. They should be a part of UX content’s strategy, not its replacement, especially if your goal is to have a competitive edge or advantage in your industry or sector. The best UX content strategies are created and designed by UX writers and content designers whose role is to craft content and strategize for human-centered experiences.

💡 Hint: AI can be helpful when it comes to strategizing. Use your AI tools to help you brainstorm, ideate, and enhance UX content strategies you want to perform. Consult with your UX writer or content designer, or book time with a professional to get expert insight.

Work with UX content experts to leverage AI

Artificial intelligence (AI) will continue to be a focal point in the tech space for the foreseeable future. So, it’s only natural for companies to lean into the new technology in hopes of being a leader in their space. When it comes to AI tools for writing, we’re biased and think writers are at an advantage to use those tools best.

Make sure your writers and content people are empowered to use AI to do their best work. Provide training and workshops and make sure they actually have time to understand what they’ve learned. They’re at an advantage when they can weave their latest learnings into their work in a way that works for them, because, subsequently, that works for the company, too.

Don’t forget to share this post:

Related articles

Why you should hire a UX content agency [Content Oyster]
UX content

Why you should hire a UX content agency

Why you should hire a UX content agency for your next project The digital world is constantly and rapidly changing, but one thing remains true: user experience is at the forefront of success for today’s leading companies. Airbnb, Apple, and Slack are just a few that come to mind. They’ve proven that if you

Content Oyster is a UX writing and content design agency based in the San Francisco Bay Area.

Copyright © 2023 – 2024, Content Oyster. All rights reserved.

Content Oyster is a UX writing and content design agency based in the San Francisco Bay Area.

Content Oyster - Elevate your web or app experience with UX writing on demand | Product Hunt

Copyright © 2023 – 2024, Content Oyster.
All rights reserved.

Scroll to Top