AI · Weekly · May 22, 2026

runtimes, not models, are where the action is

Multi-Token Prediction went from curiosity to default knob: llama.cpp added MTP, and people immediately saw 1.5–1.8× token throughput gains on commodity GPUs.[3]

On Qwen3.6 27B, MTP in llama.cpp delivers roughly a 2.44× speedup and about 100 tokens per second on tuned setups.[4] Those wins aren’t free, since MTP expands the KV cache and eats more VRAM per sequence, and some models even see slower prompt processing.[5]

On the server side, vLLM pushes Qwen 3.6 27B on a single RTX 3090 to 1261 tokens per second in prefill, then around 72.9 tokens per second in decode.[6] vLLM 0.21 layers in improved MTP for Gemma and a PreFT + pipeline-parallel prefill path for long contexts, while Vulkan-backed setups on AMD can double inference speed with RDNA2 flash attention when you’re willing to run a custom binary.[7]

chinese models are quietly eating the router

On OpenRouter, the top three models are all Chinese and together account for 58% of usage.[8] Qwen 3.7 Max comes in at $2.5 per million input tokens, undercutting US labs on price while sitting in the same router as GPT- and Claude-class options.[9]

Inference benchmarks show Qwen 3.6 27B saturating GPUs efficiently under vLLM, and NVIDIA’s DGX Spark currently holds the top speed score for serving Qwen 3.5 122B Int4 recipes.[10]

On the local side, power users are reaching for Qwen3.5‑27B variants inside Ollama for coding workloads, not just chat.[11] The combination of router defaults, aggressive pricing, and strong kernels means Qwen and peers now show up everywhere from bargain cloud APIs to local rigs, rather than as a niche “non‑Western” curiosity.[12]

agent frameworks are splitting into boring infra and magic platforms

LangGraph 1.0 has settled into a specific niche: it’s praised for bounded, deterministic workflows and called out as a bad fit for open‑ended agents.[13]

A runtime‑agnostic LangGraph/Mastra spec is now on the table, aiming to encode these graphs once and move them across runtimes.[14] Developers are explicitly comparing LangGraph with the OpenAI Agents SDK and highlighting debugging and traceability as the main differentiators, with OpenAI’s vertically integrated stack winning on convenience if you stay inside its walls.[15]

For simpler jobs, plenty of people are ripping LangGraph back out and reverting to plain Python because the graph abstraction feels heavier than the actual problem.[16]

LangGraph.js plus long‑term memory pushes the other way, dragging a chunk of orchestration into the browser and turning the client into a stateful agent host with cross‑session memory baked in.[17]

agents are powerful, expensive, and security‑toxic

OpenClaw now has around 370,000 GitHub stars, but what people are actually wiring up is a general‑purpose agent with access to highly sensitive local data.[18]

Its creator burned roughly $1.3M in OpenAI tokens in 30 days, while typical users report about $360 per month to run what they call “lightweight” agents.[19]

Projects like AutoResearchClaw show real autonomy—tool‑using agents doing end‑to‑end research—yet users still report looping on basic tasks like email drafting and problems with stale data accumulation.[20]

At the same time, one malicious VS Code extension just compromised about 3,800 GitHub repositories by stealing developer credentials, which is exactly the layer where many of these agents plug in.[21]

From August 2, 2026, the EU AI Act will treat AI agents and SaaS as regulated products, moving setups like OpenClaw from “cool script” territory into something that looks a lot more like a regulated attack surface.[22]

framework fatigue and the observability crunch

LangChain is still the default for non‑trivial flows and AI codegen, but users are clearly fatigued by rapid API churn and painful state management that keeps breaking existing material.[23]

The irony is that some of the best hard numbers still come from LangChain setups, like a RAG chatbot that got a 19% quality lift and 79% cost reduction after targeted improvements.[24]

Observability tools such as SmithDB cut trace visibility from minutes to seconds, and new monitoring projects are being built just to track what agents did and where they failed in production.[25]

Around all this, FastAPI is turning into the glue—students are shipping CatBoost MLOps pipelines, AI code reviewers, auto‑reply bots, and trading simulators as small FastAPI services wrapped around LangChain or agents.[26]

That mix of LangChain graphs, LangGraph nodes, and FastAPI endpoints means the “orchestration layer” is now scattered across frameworks and web services, which makes portability easy and ground‑truthing behavior very hard.[27]

local ai has quietly stopped being a toy

Local stacks crossed a line this month: one benchmark showed eight different LLMs running on a CPU‑only mini PC and described the experience as surprisingly usable.[28]

An RX 580 is running a full local AI server over Vulkan, and enabling RDNA2 flash attention on newer AMD GPUs can roughly double inference speed if you build a custom binary.[29]

On the UX side, Ollama is now regarded as comparable to LM Studio for coding and image generation, while Ghostbar gives vLLM‑backed models a lightweight macOS client for local deployments.[30]

Users still complain about slow generations, command‑line friction, and fuzzy safety/legitimacy of local models, but they also like that none of their data leaves the machine.[31] llama.cpp GUIs and shared configs for cards like the RTX 5060 Ti are filling in the last‑mile gaps, turning “run a 27B model locally” from hobbyist pain into a mostly copy‑paste exercise.[32]