In the last year, chatbots powered by Large Language Models are everywhere and even … useful. But how do they work?Large language models are the fundamental architecture behind chatbots like ChatGPT or Bard. A question typed in to ChatGPT, such as “What is the capital of France”, has to be processed by an LLM in order to produce an answer like “The capital of France is Paris”.
But the distances are not only expressed in two dimensions as map references, nor in three dimensions as in this cloud ...in one LLM. We don't know exactly what the dimensions encode, but they may be things like"substitutability" - on that dimension"happy" would be close to"sad", even if on another dimension they might be very far apart.
First the LLM weights all the relationships between all the words it knows, in thousands of dimensions, based on its immense corpus of training data. But then, crucially, it looks at what words have come before and reweights those associations. It is possible that within the model's computations,"Tidy" has a close relationship with"utility" and"tools" and that informs the output. .