Blog
-
Test-Time Compute Scaling with Process Reward Models: Mechanisms, Strategies, and Trade-offs
TL;DR Process Reward Models evaluate each intermediate step in a Chain-of-Thought trajectory, catching reasoning errors at their point of origin to improve accuracy. Step-level generative verification leverages explicit reasoning and code verification, allowing compact models like GenPRM to outperform much larger baselines. Offline compute strategies…
-
Kolmogorov-Arnold Networks: Rethinking Neural Architecture via Learnable Spline Activations
TL;DR Kolmogorov-Arnold Networks (KANs) serve as an alternative to Multi-Layer Perceptrons by placing learnable B-spline activation functions directly on network edges to model complex, non-monotonic dependencies. KANs significantly enhance model interpretability in applications such as cognitive diagnosis while achieving competitive training efficiency through optimized implementations.…
-
Debugging Silent Context Truncation in Ollama’s OpenAI-Compatible API
TL;DR Ollama’s OpenAI-compatible API silently trims prompt inputs that exceed the context window instead of returning an explicit error. This truncation occurs because standard OpenAI API parameters like max_tokens do not configure Ollama’s input context size. Developers can detect truncation using OLLAMA_DEBUG=1 and resolve it…
-
vLLM CUDA Out of Memory: Fixing Startup, Runtime, and Persistent OOM Loops
TL;DR “vLLM CUDA OOM” is three different problems: a startup failure (ValueError: To serve at least one request with the model’s max seq len …), a runtime failure under load (torch.OutOfMemoryError: CUDA out of memory. Tried to allocate …), and a persistent loop where every…
-
Fixing LangGraph GraphRecursionError and LangChain Agent Infinite Loops
TL;DR GraphRecursionError: Recursion limit of N reached without hitting a stop condition means a custom LangGraph run executed N super-steps without any node routing to END. N is whatever recursion_limit is set to for that run. The familiar 25 is the LangGraph JS default and…
-
Demystifying Mixture of Experts Routing: Core Mechanisms, Optimization, and Practical Challenges
TL;DR Mixture of Experts architectures scale model capacity while keeping active computational costs low by using a gating network to route tokens to specific expert subnetworks. Empirical analysis demonstrates that expert activation patterns are highly task-conditioned, with prompts from the same category producing highly similar…