Understanding Token Usage in Conversation History with LLMs
Question: Where is this held? As tokens in the LLM? (Maintains a conversation history list with message roles and content) Answer: The conversation history is held in memory as a Python list variable, not as tokens in the LLM. In the code, you can see this implementation: This is a regular Python list that contains […]