OpenAI's Jalapeño Chip Challenges Nvidia on AI Inference

FEATURED INSIGHT💡

OpenAI's Jalapeño Takes Aim at the Inference Bottleneck

OpenAI has published the first measured results for Jalapeño, its custom inference chip. On the public InferenceX suite, the company tested GPT-OSS 120B, DeepSeek R1 670B, and Kimi K2.5 1T and reported 1.5 to 1.9 times more work per watt at peak throughput and 1.7 to 3.6 times lower end-to-end latency than the commercial systems in its comparisons. These are company-reported lab results, normalized using each accelerator's published power rating; production performance at scale has not yet been demonstrated.

Jalapeño's design treats the chip, memory, network, and serving software as one system. It keeps model state such as the KV cache close to the compute that needs it, reducing the delays caused by moving data between components. OpenAI also used its models during circuit design and software development. The company says the chip went from design to tapeout in nine months, while model-assisted work brought three open-weight models to high performance in two months. Selected AI-generated kernels ran 1.5 to 1.8 times faster than earlier human-written versions, though that figure applies to specific blocks rather than complete models.

The boundaries matter. Tom's Hardware notes that Jalapeño was not compared with Nvidia's newer Vera Rubin platform and does not train models. OpenAI plans limited deployment in its own infrastructure by year-end and, according to Axios, does not plan to sell the chip to others. Builders will not choose Jalapeño directly, but inference-specific hardware could change the speed, availability, and cost structure behind the APIs and agents they use.

Join CX leaders from Clay, Anthropic, and more at Pioneer

AI is fundamentally transforming CX, making perfect experiences possible. But there's no playbook for delivering them.

Join Pioneer on October 7th to explore how CX leaders are redefining what's possible with AI.

You’ll learn how some of the most innovative leaders from Clay, Anthropic, Gamma, and more are transforming their organizations and reinventing their operations.

Plus, you’ll hear from Fin product leaders on their vision for Fin and be the first to see what’s new.

Reserve your spot

ON THE HORIZON 🌅

Perplexity Puts the Agent Stack on Your Desk

Perplexity's Portable Computer moves the machinery behind an AI agent onto hardware the user owns. On an Nvidia DGX Spark, the local system runs its orchestrator, planner, tool router, scheduler, durable task queue, and search index alongside Qwen 3.8 27B or Perplexity's post-trained PPLX 27B. It can search files, synthesize documents, execute tools in a sandbox, and keep long-running jobs active without sending the underlying work to Perplexity's cloud.

The design is hybrid rather than fully disconnected. Local work consumes no Perplexity credits, while the agent can request permission to send a specific step to cloud search, connected apps, or one of more than 15 frontier models. Perplexity says private source material can remain on the device during that handoff. Connectors for Google Drive, Gmail, Slack, and GitHub let the agent work across services, but any outbound content requires user approval.

The first release is narrow: it is available to Pro and Max subscribers on Linux with a DGX Spark, and Windows and broader RTX support are still coming. Tom's Guide reports that compatible PCs will need an Nvidia GPU with at least 24GB of VRAM. That hardware limit keeps Portable Computer out of most offices for now. Its architecture offers a useful preview of agents that separate private local work, paid cloud reasoning, and explicit permission instead of treating every task as an automatic cloud job.

LATEST IMPORTANT NEWS 📰

Apple Builds Bigger Local-AI Machines

Apple introduced the M6 and M5 Ultra for the new Mac mini and Mac Studio. The 2-nanometer M6 pairs a 12-core CPU and GPU with a dual 16-core Neural Engine, while the quad-die M5 Ultra offers up to 512GB of unified memory and 1.2TB per second of memory bandwidth. Apple says the latter can keep models with hundreds of billions of parameters in local memory. For developers handling sensitive data or repeated inference, the new systems expand the class of AI work that can stay on a desktop rather than move to a hosted GPU.

Gemini 3.5 Transcribe Targets Real-Time Voice

Google put Gemini 3.5 Transcribe into public preview through the Gemini API and its enterprise agent platform. One API streams speech with sub-second latency; another processes recordings with speaker attribution and word-level timestamps. Google reports average word-error rates of 4.0% for streaming and 2.6% for prerecorded audio, based on Artificial Analysis measurements, and supports custom vocabulary plus automatic detection across more than 85 languages. Voice-agent teams now have one model for live interaction, structured transcripts, and tool delegation, but should test accents, jargon, noise, and speaker counts against their own audio.

Claude Shares Memory Between Chat and Cowork

Anthropic has unified Claude's memory across chat and Claude Cowork, so context saved in one product can inform work in the other. Claude now adds topics to memory during a conversation, and users can inspect, edit, delete, pause, or reset the saved topic files. Memory is on by default for Free, Pro, and Max users across web, desktop, and mobile, while sensitive subjects such as health and religious beliefs are excluded unless the user opts in. The change reduces repeated briefing, but it also makes the Memory settings a practical part of reviewing what context follows an agent into new tasks.

Build a Holiday Creator Affiliate Program in 90 Days

Creators lock in holiday content calendars 90 days out, before brands figure out commissions. Waiting too long to launch an affiliate program means less runway to build demand and a missed shot at the best partnerships.

The 90-Day Holiday Sprint covers commissions, recruiting, and scaling a program at Day 30, 60, and 90.

Download the Free Guide

FOR THE TECHNICALLY INCLINED 🛠️

Tool Calls Start Before the Model Finishes Writing Code

Alex Zhang's speculative programmatic tool calling technique attacks agent latency at the harness layer. As a model streams code for a REPL, a shadow REPL parses complete, side-effect-free tool calls and starts them early; when the final program reaches the same call, it claims the in-flight or cached result. In five-run tests on OOLONG and OOLONG-Pairs with a Recursive Language Model harness, Zhang reports up to roughly a 1.2x speedup and stresses that gains vary with tool latency, generated-token count, serving load, and the model's path through the task. The MIT-licensed reference implementation marks unsafe dependencies such as file reads as ineligible for speculation. For code-generating agents, this is a way to overlap model generation and tool execution without retraining the model, provided the harness keeps speculative work pure and discardable.

AI TOOL OF THE DAY 🚀

Vercel Connect is a beta service that gives AI apps and agents short-lived, scoped credentials for services such as Slack, GitHub, Linear, Notion, and Snowflake instead of placing long-lived provider secrets in the agent's runtime.

The best voice models, now with full orchestration. Build real-time voice and chat agents on one low-latency stack: any LLM, your tools and knowledge, testing, Guardrails, and omnichannel deployment.

Start building free