How to analyze customer feedback with AI and clear criteria

Analyzing customer feedback with AI starts with reading criteria rather than a generic request to discover what everybody thinks. Comments can reveal recurring subjects, but the sample must be interpreted within its context.

This tutorial classifies fictional messages about support. The aim is to build understandable categories, preserve evidence and distinguish observations from interpretations that still require verification.

Define the question behind the analysis

Choose a concrete decision, such as finding which steps generate questions. That differs from measuring satisfaction across all customers or explaining why customers cancel.

Record where comments came from, the included period and which messages were excluded. People who respond spontaneously may have different experiences from those who never send a message.

The Nielsen Norman Group’s thematic analysis guidance describes identifying themes in qualitative material. Here we use AI to support organization while retaining review of the original texts and criteria.

Read a sample before creating labels

Select several messages you may use in an authorized environment. Remove unnecessary information and keep an internal identifier that lets you find the original again.

Fictional message	Possible theme	Observation
“I could not find my request status”	Tracking	Question about location
“They replied quickly but did not explain the deadline”	Deadline and support	Contains praise and criticism
“Everything fine”	Insufficient detail	Does not explain a step

One message can address several subjects. Forcing it into a single label may hide precisely the combination that interests the team.

Write criteria somebody else can apply

For each category, record a definition and an inclusion example. Also specify what belongs elsewhere and how ambiguous messages should be handled.

“Experience problem” is too broad to guide an action. “Difficulty finding request status” points toward an interface or communication worth reviewing.

Include a provisional category for insufficient or unclassified material. This prevents AI from inventing meaning just to complete the table.

Do not refine the labels around a single memorable comment. Check whether the same definitions work across varied messages before applying them to the full collection.

Request classification with evidence

Supply category definitions and an identifier for every message. Request a brief justification supported by a passage from the comment itself.

Classify these fictional messages using the supplied definitions.
A message may receive multiple themes.
Return its identifier, theme, evidence excerpt and any interpretation question.
Do not infer age, cancellation intent or overall satisfaction.
Use “insufficient information” when the comment cannot support a decision.
Do not invent statements or present paraphrases as literal quotations.

Compare some outputs with an independent reading. If two reviewers repeatedly disagree, the category definition may need improvement before expanding the batch.

Count without overstating the conclusion

State whether the counting unit is message, customer or theme occurrence. When one comment has two themes, occurrences may outnumber comments.

A complaint count in a sample does not automatically measure the proportion of dissatisfied customers. Keep the denominator explicit and describe how messages were selected.

Use “in the analyzed sample” when that is the actual scope. This keeps findings useful without turning them into a statistic unsupported by collection.

Turn a theme into a next investigation

Choose one subject and connect it with an operational question. For example: on which screen did people expect to find their request status?

The work is ready when every conclusion can be linked to messages and applied criteria. Theme popularity alone does not determine product priority.

Keep representative evidence and counterexamples together. A theme may describe a real problem without explaining every customer’s experience or the reason it occurred.

Frequent errors include mixing periods, counting overlapping categories as unique people and asking AI to guess motivations. To review incomplete export fields, see the missing-values guide.

Frequently asked questions

Can one message receive multiple themes?

Yes, when it covers different subjects. Explain occurrence counting so it is not confused with unique customers.

Does counting complaints measure all customers’ satisfaction?

Not automatically. Scope depends on collection, sampling and the denominator used.

How can I check AI classification?

Compare labels with definitions and original passages, including independent reading and ambiguity handling.