LLM Evaluation

We propose a causal representation learning framework to understand the hierarchical structure of language model capabilities, revealing causal relationships between general problem-solving, instruction-following, and mathematical reasoning abilities.

We build a novel benchmark for evaluating how well can state-of-the-art LLMs solve inequality proving problems. Our evaluation studies provide insights on their advanced math reasoning capabilities and highlight their common reasoning flaws.