Code editor company Cursor is facing user backlash after its AI-powered customer service chatbot fabricated a company policy. The incident highlights the potential pitfalls of relying too heavily on AI for customer interactions. During a recent support session, the bot conjured up a previously nonexistent rule, sparking confusion and anger among users who immediately challenged its legitimacy and questioned its implications. This episode underscores the limitations of current AI models in accurately comprehending and representing complex company policies. The resulting ‘hallucination’ has raised concerns about user trust and the reliability of AI-driven customer support.

Photo by luis gomes on Pexels
Photos provided by Pexels