Neural networks don’t just memorize patterns—they can grow their own symbols!  This paper for the first time proves that, under regular training, the weight distribution naturally collapses onto lower-dimensional “manifolds” that behave like algebraic building blocks (think little logic gates) and then learns to compose them. By casting training as a smooth flow of probability mass and showing it splits into independent, ring-structured sub-tasks, the authors give the first rigorous account of how continuous gradients can birth discrete reasoning—and even derive width “scaling laws” that say how big a net must be to master a given symmetry or logic rule.

​SPIN-Bench stress-tests today’s chatbots on what makes strategy games hard for people: thinking many moves ahead and reading (or bluffing) other players. It bundles classic planner puzzles with board games like Chess, the cooperative card game Hanabi, and negotiation-heavy Diplomacy, then shows that even top LLMs stumble once action trees get wide or alliances shift — often scoring well below novice humans in social settings.

SEAL breaks a language model’s chain-of-thought into execution, reflection, and transition steps, then builds a single “steering vector” in latent space that dampens the meandering reflection/transition phases and keeps the model on the direct execution path—no fine-tuning required. Plugging this vector in at inference boosts accuracy by up to 11 % on math, coding, and logic tasks while cutting the number of generated tokens — and therefore time and GPU memory — by 12–50 %.

LLM-AutoDiff treats an entire LLM workflow — symbolic engines, retrievers or other tools, even cyclic agent loops — as one auto-differentiable graph where every prompt is a “parameter” and a frozen back-ward LLM supplies textual gradients that flow through the graph to rewrite only the components that caused the error. This language-level back-prop cuts the prompt-engineering grind to a handful of mini-batches, boosting accuracy by up to ~10 percentage-points on multi-hop HotPotQA and other benchmarks while consuming far fewer tokens than Text-Grad or DsPy. In short, it gives prompt engineering the same one-click optimize button that PyTorch gave neural networks, turning messy multi-LLM pipelines into self-tuning systems.

We reveal that a robot’s miscues come from two very different places—what it sees versus how it plans—so the authors split uncertainty into perception and decision parts, then quantify each with neurosymbolic theory-backed tools (conformal prediction for “continuous” vision, formal-methods-driven checks for “discrete” plans). Once the model can measure its own blind spots, it triggers active re-looks at confusing scenes and fine-tunes only on high-certainty data, cutting output variability by 40 % and pushing rule-compliant driving from about 60 % to 95 % in both CARLA simulation and real-world tests.

We replace costly human feedback with formal-methods feedback: each text policy an LLM proposes is auto-translated into an automaton and model-checked against traffic-rule specifications, and that pass/fail signal steers a DPO fine-tune. Looping this symbolic verifier in for Llama-2-7B on driving tasks lifts spec-compliance from 60 % to 90 %, needs only a few thousand prompt pairs, and the resulting controllers run unchanged in the CARLA simulator — pointing to a verifiably safe alternative to RLHF for autonomous systems.