Project description
GESIS – Leibniz Institute for the Social Sciences
2024 – ongoing
Conversational commerce promises a more natural way to shop, but handing decision-making over to a large language model raises a hard question: can a shopper trust a recommendation they can't inspect? Cleo answers this with three design contributions. First, transparency: the system prompts the LLM to reflect on its interpretation of the user's needs, while an auditable ranking mechanism exposes the loss values behind each recommendation, so a shopper can see why a laptop was suggested, not just that it was.
Second, controllability: a hybrid architecture separates a deterministic ranker, which applies filters and numeric scoring across thousands of product specifications, from a constrained language model that generates descriptions grounded strictly in catalog evidence. This keeps the fluency of conversation without the risk of hallucinated or persuasive claims.
Third, decision support: natural-language comparisons and a highlights feature contextualize specifications relative to what the user actually asked for, so shoppers can evaluate options without parsing raw specs line by line.
Under the hood, Cleo runs as two independently deployable services: an Angular/TypeScript frontend and a Python/Flask backend. The frontend also supports voice input through the browser's Web Speech API, transcribing speech to text on the fly with no external speech service. The backend calls the OpenAI API directly through a custom prompt-management and conversation layer, and computes recommendations with a deterministic scoring engine built on PyTorch tensor operations (fast tensor math rather than a trained model) that ranks the catalogue against each user's stated requirements. Each user turn is routed through a constrained tool-calling layer. The model expresses intent through a fixed set of five structured calls (answer review questions, answer a laptop follow-up, compare laptops, update requirements, request new recommendations), can emit several in one turn, and can only target laptops currently on screen. An explicit ambiguous option triggers a clarifying question rather than a guess.
