Beyond probability
An LLM is basically a massive free-association machine, and we’re giving them more data & compute, but it’s still using probability instead of advanced reasoning. I guess probability is like a low-complexity, scalable form of reasoning. To advance, it has to be able to read paragraphs, run a chain of thought (what they currently do), but then compress that thought into symbolic logic (meaning they’ll reason forward with variables, formulas, and algorithms, not just % likelihood).
AI in the 60s-90s tried this approach, but it was too limiting. Maybe it’s too complex to design a single “thinking algorithm” and instead it needs an engine to derive it’s own algorithms for thought, and once you have that, I think that leads to something like a takeoff.
“That framing captures a key reality—LLMs function as large-scale probabilistic sequence learners that free-associate based on statistical patterns rather than explicit symbolic manipulation, and emergent chain-of-thought prompting can approximate multi-step reasoning “Chain-of-Thought”, but they do not natively compress reasoning into interpretable symbolic logic without hybrid modules Rebooting AI; historical symbolic AI efforts like CYC showed that fixed rule-based systems struggled with linguistic complexity, suggesting that future progress may require architectures that derive and refine their own internal algorithmic primitives—combining probabilistic prediction with learned symbolic abstractions—to unlock genuinely advanced, self-amplifying reasoning.
Research into neurosymbolic AI—integrating large-scale neural pattern learning with symbolic reasoning—continues at institutions like the MIT-IBM Watson AI Lab and DARPA’s ANSR program, with early prototypes such as the Neuro-Symbolic Concept Learner demonstrating limited vision-language reasoning and IBM Research championing hybrid architectures for interpretable, compositional models; whether explicit symbolic modules are required for robust AGI is debated—while scaling laws and chain-of-thought prompting show statistical learners gaining some reasoning capacity, others argue that true algorithmic understanding demands symbolic scaffolding. If meta-learning engines that derive their own reasoning primitives emerge, recursive self-improvement could swiftly transition AGI into superintelligence; optimistic forecasts (e.g., Shane Legg’s 50% by 2028) contrast expert medians around 2047–2060 and superforecaster central estimates near 2070, implying that an ASI takeoff could follow within a few years of AGI—though timelines remain highly uncertain.”