WORLDTECH NEWS Global technology intelligence.Contact
← Back to WORLDTECH
AI SINGLE SOURCE

Spelling backward gets easier for AI models upgraded to see every letter

Close-up of a smartphone showing popular social media apps on screen.
Illustrative photo.Photo by Atlantic Ambience on Pexels

What happened

Before large language model (the kind of AI system trained on text to produce text)s (LLMs) can process text, they split it into chunks such as words or word fragments. The LLM behind ChatGPT, for example, splits "LMU München" into the three chunks "LM," "U" and "München," so it never directly sees the individual letters in "München." This step, which is known as subword tokenization (representing an asset as a tradable entry on a blockchain), makes LLMs efficient but causes a range of problems, such as limited character-level understanding.

Sources & evidence