The Geometry of Categorical and Hierarchical Concepts in LLMs

Published by

yjchoe.github.io

on •1 min read

The Geometry of Categorical and Hierarchical Concepts in LLMs

Understanding how semantic meaning is encoded in the representation spaces of large language models is a fundamental problem in interpretability. In this paper, we study the two foundational questions in this area.

Linear Representations Representation Geometry Softmax Geometry Causal Inner Product Large Language Models Gemma Llama Wordnet

Learn more