What is Unembedding matrix? and how is it different from an embedding matrix?
9:25 am · Aug 30, 26 · 17 follow-ups
What are vectors and what are tokens in a large language model?
1 follow-up
What is a Transformer, and why is it the key architecture behind LLMs?
What happens to a token after it enters the Transformer?
What does a word embedding actually represent? How many numbers are generally assigned for an average word like 'Cat' or 'Dog'?
Can embedding matrix represent things beyond single word meaning or even a deeper understanding of language as a whole given it contains more tokens than any human mind??
If different layers of a neural network end up doing different things, who decides what each layer should learn? Are the roles of the layers assigned manually, or do they emerge automatically through training?
How are these vectors converted into tokens for final output?
What is temperature, and how does changing it affect the generated text?
2 follow-ups
What does 'learning' mean for this neural net, and how does the system learn by self organization of these vector embeddings?
