Content Peep · Weekly · May 13, 2026

local-first inference is fast enough to matter, fragile enough to hurt

Everyone is posting tokens/sec screenshots, but the interesting part is that Qwen 3.6 27B is now reported at up to 135 tokens/sec on a single RTX 3090 via BeeLlama.cpp, putting local agents into 'production-ish' territory.[1]

The same model is also hitting around 80 tokens/sec at a 128K context window on a 12GB GPU, which changes what a solo builder can do with on‑device coding agents.[2]

Gemma 4 26B is clocking about 600 tokens/sec on an RTX 5090 in optimized runtimes like vLLM.[3] DeepSeek V4 Flash reaches 85.52 tokens/sec at a 524K context window on dual RTX PRO 6000 Max‑Q GPUs while costing roughly 90% less than GPT 5.4 Mini.[4]

The catch is that all of this is extremely stack‑sensitive: builders report llama.cpp slowdowns on Windows with AMD GPUs, Vulkan generally beating ROCm but needing careful tuning, Qwen 3.6 instability in some harnesses, and Ollama both struggling with complex reasoning and exposing an unauthenticated 'Bleeding Llama' memory leak with potential remote code execution.[5]

decoding tricks are now part of the product spec

Speculative decoding methods like DFlash are delivering up to 8.5× faster generation on some LLMs without measured accuracy loss in reported benchmarks.[6]

On an RTX 5090, DFlash is part of the stack that lets Gemma 4 26B reach around 600 tokens/sec, and users report it outperforming classic MTP on parallel block diffusion drafting and stateful context management.[7]

Multi‑Token Prediction gives Qwen 3.6 27B about a 2.5× speedup over baseline decoding, and the Qwen 3.6 35B A3B variant is generating around 80 tokens/sec at a 128K context on 12GB VRAM.[8]

Gemma 4 MTP builds are drafting roughly 40% faster than standard LLaMA.cpp‑style decoding in community benchmarks.[9] The other half of the story is where it breaks: DFlash often degrades beyond ~20K tokens and seems best under about 4K tokens on some models, MTP eats more VRAM, and gains are much weaker on creative chat than on coding or tool‑using agents, prompting calls for workload‑specific validation suites.[10]

agents are turning into distributed systems

Hermes Agent is now the most‑used AI app globally on OpenRouter, surpassing Claude Code and OpenClaw in token volume, and is being wired into cloud services that can create Cloudflare accounts, buy domains, and handle payments via USDC on AWS.[11]

At the same time, there’s a visible shift toward personal and self‑hosted agents, with trends away from subscriptions to locally hosted setups and new releases like n8n‑as‑code V2 embedding an agent directly into VS Code for workflow management.[12]

Telegram is becoming a de facto agent surface, adding Guest AI Bots, bot‑to‑bot chats, chat automation, and Telegram‑Drive while bots on the platform already handle voice notes, images, chess analysis from screenshots, and CRM‑style workflows.[13]

Under the hood, orchestration is shifting from opaque single agents to explicit graphs and flows, with LangGraph 1.2 adding delta channels and checkpointing for long‑running agents and n8n powering multi‑layer AI revenue‑intelligence and fraud‑detection systems across Redis, PostgreSQL, and LLM agents.[14]

MCP is emerging as a standard protocol layer in these stacks, standardizing how agents discover and call tools while also acting as a security boundary and shared auth layer across multiple services.[15]

memory and rag are becoming their own infra layer

Anthropic’s agents now use a sleep mechanism to replay experiences and reorganize memory traces, while the Hermes Memory Installer 2.0 builds long‑term agent memory on PostgreSQL to give assistants durable, queryable history.[16]

OpenCode‑based coding agents add persistent memory so they can retain project context without re‑explanation, and separate projects use PostgreSQL to track per‑session budgets as agents query production databases.[17]

On the retrieval side, EnterpriseRAG‑Bench is being introduced to stress‑test RAG systems on complex enterprise data rather than toy Q&A logs.[18]

Blockify‑style corpus optimization reportedly shrinks document stores by about 40× and cuts tokens per query by roughly 3× compared to naive chunk‑and‑embed setups.[19]

The darker story is that memory poisoning and prompt‑injection remain common control failures in RAG agents, while MCP‑based multi‑server setups introduce a 'context tax' where tool catalogs and server metadata eat context window and degrade model behavior.[20]

ai coding just became majority author, and the backlash is starting

Airbnb reports that AI now writes about 60% of its new code, with even engineering managers using Claude Code to contribute.[21] Google says AI now generates roughly 75% of its new code, while Microsoft puts its figure at up to 30%, so in many large shops the default author is already a model.[22]

Hermes Agent has become the most‑used AI on OpenRouter, surpassing Claude Code and OpenClaw in token processing, and Codex is being run in fully autonomous modes where it completes paid bug‑fixing and security work without direct human steering.[23]

The backlash line is forming: GitHub reversed a move to make Copilot a co‑author on every VS Code project after consent and job‑security fears, usage‑based Copilot billing is confusing developers, and audits of Lovable/Replit‑built apps have found thousands of deployments leaking credentials or exposing sensitive data.[24]

In the trenches, Cursor‑style 'vibe coding' workflows are speeding up delivery but leaving teams to wrestle with hallucinated agents and code‑quality worries, while veteran programmers argue that AI still cannot replace the need for humans who understand security and architecture even as most job postings barely mention AI skills.[25]