TAG
#inference
Stories connected through a shared idea, actor, technology or theme.
DeepSeek open-sources its production inference engineDeepInfer supports NVIDIA SM100 GPUs and Huawei Ascend 950 NPUs, with optimisations for scheduling, speculative decoding and large-model execution. The release does not include DeepSeek’s full cache-management, prefill/decode-disaggregation or load-balancing stack.→SGLang 0.5.22 delivers faster model loading and a critical state-correctness fixSGLang 0.5.22 introduces an InstantTensor loader that maintainers say makes models load up to ten times faster, adds new model and diffusion support, and improves DeepSeek-V4.1 throughput on H200 GPUs. It also fixes a bug that could resume hybrid-linear models from incorrect Mamba states when speculative decoding and structured output were combined.→vLLM 0.31 adds faster restarts and broader large-scale serving supportvLLM 0.31 introduces a weight-cache daemon and snapshot create/restore workflows for faster engine recovery. The release also expands speculative decoding, large-scale expert-parallel serving and DeepSeek-V4.1-Flash optimisations.→SiMa.ai raises $150 million to scale purpose-built physical-AI chipsThe round brings SiMa.ai’s total funding to $500 million and will fund its Palette Neat agentic environment and next-generation hardware. The company is targeting 1,000 dense TOPS for chips aimed at humanoid robots, drones, vehicles and industrial systems.→SEMIFIVE wins a $52 million contract for a next-generation AI inference acceleratorThe accelerator will combine LPDDR6, PCIe Gen5 and large-die technologies, with tape-out targeted for the first half of 2027. Global mass production is planned for 2028, with hyperscalers and cloud providers as target customers.→Anthropic launches Claude Opus 5.5 with lower operating costsAnthropic says Opus 5.5 performs around the level of Fable 5.1 on most work while costing 40% less than Opus 5. The model has a one-million-token context window, adaptive thinking and availability across major cloud platforms.→EUCLYD raises more than €200 million for efficient AI infrastructureEUCLYD announced a Series A financing round exceeding €200 million, co-led by Samsung, Somerset Capital Partners and Scaleup Europe Fund. The company is developing specialised silicon, memory architecture and data-centre systems intended to reduce the power and cost of AI inference.→Fireworks schedules retirement of older serverless model endpointsThe notice applies to serverless, Fast and US-only serverless endpoints while leaving dedicated deployments unaffected. Fireworks lists replacement models for the affected DeepSeek V4, GLM 5.2, Muse Glimmer 30B and Kimi K2.x variants.→llama.cpp improves parsing for Qwen3-Coder complex typesThe release improves parsing of complex types in Qwen3-Coder chat interactions and adds corresponding parser support. It was published as a nightly build of the C/C++ inference runtime.→Google makes Gemini video analysis goal-directed and more efficientGoogle launched agentic video understanding across Gemini 3.7 Flash, 3.6 Flash and 3.5 Flash-Lite in the Gemini API, AI Studio and Gemini Enterprise Agent Platform. Google reports up to 88% lower token consumption, 66% lower analysis cost and 7% higher accuracy on its standard benchmarks, with Gemini app and Ask YouTube rollouts planned.→Broadcom introduces VMware Private AI Cloud for private inference and agentsThe platform combines VMware Cloud Foundation with heterogeneous accelerator support, NVMe memory tiering, storage deduplication, token monitoring, multi-tenant model sharing and GPU observability. Broadcom also says customers can access more than 150 validated open and commercial models through integrated deployment and operations services.→Think launches hosted heterogeneous AI compute with Think GridEach initial SuperNode combines four NVIDIA PRO 6000 Blackwell accelerators with 384GB of pooled VRAM, networking, storage, cooling and support. The service is available to Saudi government and enterprise customers from Riyadh and is being rolled out globally for inference, training and agent workloads.→GLM-5.3-Flash launch exposed a cross-runtime reasoning-efficiency problemSGLang said GLM-5.3-Flash initially generated about twice as many reasoning tokens on open-source engines as on Z.ai's API on AIME and GPQA, despite similar scores. Fireworks, vLLM, SGLang and Z.ai then reran tests and updated the serving path; SGLang said the excess was concentrated in some reasoning prompts and was not seen on the cited agentic benchmarks.→MiniMax reports rapid platform growth alongside heavy lossesMiniMax reported first-half revenue of US$116.6 million, up 283.1% year on year, while revenue from its Open Platform and other AI enterprise services rose 703.1% to US$73.9 million. Gross profit increased sharply, but adjusted net loss reached US$293.0 million, showing that commercial scale and compute-heavy spending are expanding together.→OpenAI publishes first benchmark results for Jalapeño inference chipOpenAI says Jalapeño delivered higher throughput, lower latency and better energy efficiency than comparison systems when serving GPT-OSS 120B, DeepSeek R1 and Kimi K2.5 1T. The results are vendor benchmarks for inference hardware, not training, and OpenAI says limited systems will be deployed this year.→Perplexity launches a local-first Computer agent for Nvidia DGX SparkPortable Computer is optimised for Nvidia DGX Spark and can run local models, including a post-trained Qwen 3.8 27B variant, while supporting connectors for Gmail, Drive, Slack and GitHub. Perplexity also allows users to switch between local and cloud models, with local execution avoiding token limits and keeping routine workflows on-device.→Nvidia doubles entry-level edge compute with Jetson Orin Nano 2Jetson Orin Nano 2 provides 78 TOPS of AI compute, 8GB of memory and an eight-core CPU, with Nvidia claiming twice the compute and 40% lower power use than its predecessor in a 15-watt mode. The module and developer kit are scheduled for general availability in the first half of 2027, with drone and industrial-robotics partners already exploring deployments.→DeepSeek removes peak-hour API pricing on weekendsThe update changes the peak/off-peak schedule introduced earlier in August so Saturdays and Sundays use the lower off-peak rate. Weekday peak pricing remains in place, making this a targeted capacity-management change rather than a blanket price cut.→SGLang and AntLing cut trillion-parameter restart time to 32 secondsAntLingAGI and Alibaba reported 0.63-second weight loading for the Ling-2.6-1T FP8 model, up to 780 times faster than disk loading. The end-to-end engine restart fell to about 32 seconds by avoiding repeated disk reads.→GLM-5.3 doubles its reported KernelBench-Mega speedupThe result compares with 11.1 times for GLM-5.2 and reportedly passes the benchmark's single-launch gate that the earlier version failed. The implementation used Kimi-Linear Decode on an RTX PRO 6000 and CUDA techniques including persistent CTAs, Int4 dequantisation and fused latent-attention handling.→Denmark's Gefion supercomputer underpins a sovereign AI control layerThe platform provides an OpenAI-compatible API for managing and deploying leading models while retaining control over compliance, security and infrastructure choices. It runs on DCAI's ISO-certified Gefion supercomputer, with Trifork as an early adopter and DCAI investing in a Blackwell B300-based inference cluster.→Cerebras unveils CS-4 rack-scale AI systemThe Register reported that CS-4 uses the WSE-3T/Turbo, which Cerebras says doubles compute, memory-fabric and I/O bandwidth while retaining its 5nm process and 4 trillion transistors. Cerebras claims up to 4,400 tokens per second per user on gpt-oss-120b, with first systems due later this quarter.→Relativity Networks raises $22 million for faster data-centre fibreRelativity Networks announced $22 million in SAFE funding and a further $40 million order from an unnamed hyperscaler, TechCrunch reported. Its hollow-core fibre is designed to cut propagation latency to about 3.5 microseconds per kilometre, compared with roughly 5 microseconds for conventional fibre.→Etched raises $700 million at a $21 billion valuation for specialised inference systemsEtched announced a $700 million round at a $21 billion valuation, led by Jane Street, roughly doubling its reported valuation in a month. The company is building full inference systems around a specialised prefill chip and cluster-scale memory, claiming the systems can run frontier models.→Groq raises $350 million as it pivots from AI chips to neocloudGroq raised $350 million at a $3.5 billion valuation, with Disruptive leading and Nvidia planning to participate. The company said it will expand its neocloud capacity from 54 megawatts to more than 200 megawatts in 2027.→AMD unveils Helios rack-scale system for frontier AIAMD introduced Helios alongside new data-centre hardware and named OpenAI, Meta, Oracle, Anthropic and Microsoft among planned users. The system is designed to combine large numbers of processors and accelerators for high-scale AI workloads.→Runway launches a routing layer for generative-media modelsRunway Media Router automatically selects among available media models according to preferences such as quality, speed and cost. The service extends Runway’s developer platform, which already provides access to its own and third-party models.→Etched reaches a $10.3 billion valuation with specialised inference hardwareEtched’s Series C was led by Sequoia, with participation from Andreessen Horowitz, SK Hynix and Jane Street. The company says it has manufactured its chips, is testing complete systems with customers and has booked $1 billion in orders.→