← 返回

2026.09.24 | Substack | LLM | FUNDA: Jev Users Report Savings

原标题：Deep|LLM: Jev Users Report 10× Faster and 54.5× Cheaper Than the Models They Replaced; Only 3.7% in Production

#llm

Substack · FUNDA · 2026-09-24

Jev is a “decision model” from TypeSafe AI, released September 15, 2026 and opened to all users on September 20. It does not generate text. It answers questions with a fixed answer set: pick an option, score on a scale, or judge true/false, and attaches a confidence score. The launch quickly gathered industry interests, and some investors were asking whether it’s a significant negative to compute demand. As we addressed in our report earlier, we disagree with that concern and believes Jev is more of an interesting trial with limited impact on LLM.

1:14 AM

On September 22, Anthropic launched Claude Opus 5.5 at $4 per million input tokens and $20 per million output tokens, 20% below Opus 5. Anthropic estimates that a typical customer workload costs 40% less to run. OpenAI launched GPT-6 Sol ($2/$10) and GPT-6 Luna ($0.10/$0.50) the same day, at prices 50–58% below the prevailing rates for the corresponding GPT-5.6 models and 60–92% below their July 9 launch prices. GPT-6 Luna is the first U.S. model since DeepSeek launched its V4 series to undercut DeepSeek’s cheapest model on both input and output list prices: DeepSeek V4.1 Flash costs $0.15/$0.60 even at off-peak rates. In the text, paired prices such as $4/$20 denote input/output charges per million tokens.

To analyze Jev further, we decided to have a deep dive into what Jev use cases are really about. This note covers 6,277 public discussions and use cases from the first 7 days; 2,153 are from people who actually used or tested it.

1. What developers use it for

In week one, public discussion was mostly opinion; few people had actually used or tested Jev. Opinion and analysis are

  1. 8% of the 6,277 counted cases (Figure 1). The 2,153 hands-on cases are the better read on demand; the rest of this

note uses that cut.

Figure 1 All 6,277 cases by type of content

Source: FUNDA

Among hands-on cases, the largest uses are real-time control, agent control decisions and content classification. None reaches 20%. They are 18.8%, 16.1% and 15.7% of the 1,284 hands-on cases with an identifiable use (Figure 2). Table 1 shows a typical case for each.

")

Figure 2 The 1,284 hands-on cases with a specific use, by use (color shows the broader group)

Source: FUNDA

An open-source browser automation project uses Jev at every step to decide what to do next and which control to click or fill, and calls a small model only when it needs to type. One flight search took 7.1 seconds end to end: that is whole-task time, not a single decision. “Give it one goal. TypeSafe’s Jev picks an operation and an element. A small LLM writes text only when the operation is TYPE\_TEXT”

In real-time control, someone built a Tesla self-driving control demo with Jev in under an hour. Build time only; no other figures. Still a demo.

Shared traits: preset answer list, frequent calls, speed matters. In the 2,153 hands-on cases, Choice (pick one of the given options) is the most common question type at 23.0%; Noul (true/false) and Score (scale) are less common (Figure

  1. . Against the vendor’s own use list, real-time applications and branching decisions in code are 28.9% and 27.8% of the

1,284 identifiable-use cases. Bulk data processing is 13.2%: not a main use today.

")

Figure 3 Question types and vendor-recommended uses in hands-on cases (a case can have several)

Source: FUNDA

2. What Jev replaces

Jev does the deciding inside a program, not the writing. Classification and routing is the largest group among the 1,284 hands-on cases with a clear purpose (27.0%), then real-time and games, then search and ranking (Figure 4). These steps choose from preset answers and run many times; text generation does not matter there.

Figure 4 The 1,284 hands-on cases with a specific use, by broader group

Source: FUNDA

Most of what Jev replaces is frontier models, though more than six in ten hands-on cases never say what came before. Among the 2,153 hands-on cases, prior approach is unknown in 62.1% and a frontier model in 15.1%. Rules or heuristics (keyword matching and similar) are 6.6%; human review is 0.7% (Figure 5). When Jev’s confidence is low, the most common fallback is a person (7.4%); 7.8% have no fallback at all (Figure 6).

Figure 5 What Jev replaced, and the models developers compared it with

Source: FUNDA

")

Figure 6 What happens when Jev is not confident enough (hands-on cases)

Source: FUNDA

Subscribed

In one tax-filing workflow, Jev took over page classification from Claude Sonnet. One call per page, pick from 261 IRS form types and 7 page types, return a confidence score; below 0.95 counts as error. On a 314-page test file the strict error rate was 0.00%, at about $0.00115 per page; the system now runs in production. “Jev classifies 100% of our tax document corpus at $0.001 per page — 34× cheaper and 6× faster than that LLM setup. This is the classifier, open sourced”

Developers compare Jev with cheap small models almost as often as with the strongest large ones. That is a clearer read on the moat than the volume of criticism. Of the 432 cases that name a comparison model, 48.6% mention a small or open model and 59.7% a frontier model (Figure 5). Only 17 cases have moved a live service to Jev; the models they replaced are a mixed group (Table 2).

A product that routes coding tasks across AI tools moved that routing decision in production from a large model to Jev. Jev now picks the tool, the model and the reasoning effort in about 200 ms. It matched Claude Fable 5.1’s choices on all 25 test tasks and cut routing cost 71% and time 90%.

3. Cost and speed

Almost all price figures in the discussion just repeat the vendor. Users’ own numbers cluster on latency, total run cost and cost per call. Of 170 figures for price per million tokens, 156 repeat the vendor; none is a user measurement. Of 787 latency figures, 288 are users’ own: the largest user-evidence set (Figure 7).

")

Figure 7 Where the cost, speed and quality numbers come from (each row covers all numbers of that type)

Source: FUNDA

Users measure about 10× speed-up, below the vendor’s 193.6× peak; the gain tracks what Jev replaced. Median user-reported speed-up is 10× (n=72): 10× vs frontier models, 5× vs small hosted models. In the 16 cases with latency for both, Jev’s median is 300 ms (inside the vendor’s 70–500 ms band) vs 2,924 ms for the old system (Figure 8).

")

Figure 8 Users’ own numbers vs vendor claims: latency, speed-up and cost multiple (log scale, one dot per number)

Source: FUNDA

Vercel tested Jev on safety review in its own CLI: a model decides whether each command is safe to run. Jev was about 5–18× faster than the Luna 5.6 model then in use, and more accurate. Vercel said it would add Jev to its model-calling service and might make it the default.

Savings track what Jev replaced: 100×+ vs frontier models, under 20× vs small hosted ones. Median user-reported cost multiple is 54.5× (n=56): 188× vs frontier, 17× vs small hosted (Figure 8; full figures in Table 3). Vendor comparison with GPT-5.6 Terra is about 76×. Bryo, a partner sorting business email with Jev, reports 10–20×. The 444.6× in marketing is a peak against the most expensive model; do not budget off it.

Independent evaluations mostly find Jev cheaper and faster; accuracy is mixed. More than eight in ten cases that assessed cost or latency rated Jev better. On accuracy, 83 of 241 rated it better and 65 worse (Figure 9). In the 17 head-to-heads with accuracy for both, median gap is 1.6 percentage points.

")

Figure 9 Verdicts of independent evaluations (share of cases that assessed each aspect)

Source: FUNDA

One benchmark ran Jev on 1,565 German and English business emails across 10 categories; accuracy came in below Gemini. The tester argued error location matters more than the headline rate, and was still weighing production use.

4. Calibration: how reliable is Jev’s confidence

On common tasks developers tested themselves, Jev’s confidence is off actual accuracy by about 10 percentage points on average. ECE (expected calibration error: average gap between stated confidence and actual accuracy; 0 is perfect) is the usual measure. It matters for routing: auto-run high-confidence answers, send low-confidence ones to a person. Median user ECE is 0.097 (n=27); the middle half sits between 0.0588 and 0.154 (Figure 10).

and an independent test on unfamiliar tasks")

Figure 10 User-measured calibration error (ECE, lower is better) and an independent test on unfamiliar tasks

Source: FUNDA

On unfamiliar tasks unlike the training data, the overall gap is similar, but Jev is clearly overconfident on rating questions and errors point different ways by question type. One independent test measured overall ECE 0.107, close to the users’ median 0.097. Score questions (4-level priority) hit ECE 0.325; Choice leaned overconfident; Noul understated confidence. One confidence threshold for handoff will not work; set it per question type.

One independent test used 900 rule-generated support tickets (300 each Choice, Score, Noul) to check whether Jev’s probabilities hold. On three public benchmarks the probabilities needed almost no correction. On the 900 tickets with unfamiliar rules, overall ECE was 0.107: 4.4× the smallest error the test can detect (0.024). Score questions were right only 44.7% of the time.

“Calibration is proven” and “calibration is unproven” both overreach. Average gap is about 10 percentage points; on some question types in unfamiliar tasks Jev is clearly overconfident. Across all discussion, 7.4% of cases cite calibrated probabilities as a strength and 2.2% call calibration unproven or the method undisclosed. Of 64 evaluation cases that assessed calibration, 28 found Jev better and 21 worse. Task and question type decide the answer.

5. Who is talking about Jev

Nearly everyone talking about Jev is a developer: mostly indie developers and dev-tool authors; almost none at big companies. Among the 2,140 cases where we can tell the author’s role, indie developers are 35.9% and big-company engineers 2.7% (Figure 11). Another 65.9% of all discussion gives no clue who wrote it. A consumer product looks different: of 654 Meta Muse use cases we analyzed earlier, 62.8% were everyday chores.

")

Figure 11 Authors whose role is known (2,140 cases)

Source: FUNDA

Framework and dev-tool authors are the most positive; tech journalists and ML researchers the least. Among framework and dev-tool authors, 75.4% of posts are positive, vs 17.0% for ML researchers and 2.8% for tech journalists (Figure 12). Startup engineers, CTOs and founders are 65.6% positive but also 14.7% negative or mixed: the widest split among developer groups.

")

Figure 12 Attitude by type of author (groups with at least 30 cases)

Source: FUNDA

The most shared items are opinions and news. What people actually built is mostly small demos. Near the top of Table 4: an idea to use Jev for context compression, broad takes on industry impact, and Vercel’s announcement that Jev is free for a limited time on its model-calling service. Built artifacts are mostly demos: quick website classifiers, real-time spam detectors. First-hand accounts after using or testing Jev are 33.4% of all discussion.

6. Ecosystem and market response

Discussion grew for four days after launch and peaked the day before open access. New cases rose from 275 on launch day (September 15) to 1,346 on September 19, then 1,190 on September 20 when Jev opened to all. September 21 covers only the morning, so the drop is not a read on interest (Figure 13). First integrations through Vercel, LangChain, LlamaIndex and others landed September 16–18 (Table 5).

")

Figure 13 New cases per day by first-seen date (UTC; the last day counts only the morning)

Source: FUNDA

Most developers call the API directly. Each tool integration has a small share. Queues and outages stopped almost no one. Of 2,153 hands-on cases, 33.9% call the API directly; no other route tops 6.1% (Figure 14). Across all discussion, only 0.2% mention being kept out by the waitlist.

Figure 14 How developers connect to Jev, and access problems

Source: FUNDA

Open-source replicas showed up in volume within a week: the approach is easy to copy. On price, first competitors are cheap small models. Clear replicas are 3.0% of all discussion (Table 6 lists the main projects). Most share a pattern: existing open model + decision module + Jev’s three question types, run on a PC or a single GPU. Vendor evaluation puts Jev level with GPT-5.6 Terra on accuracy (67.8% vs 67.9% agreement with reference answers) at about 76× lower cost. But

  1. 6% of cases that name a comparison model put Jev against a small or open model, so price competition starts in cheap

classification and routing.

One replica pairs a small open model with a decision component, supports Choice, Noul and Score, and publishes training process, weights and data. On a shooting-game task it scored 128/128 vs Jev’s 56/128, but it was trained on those tasks and Jev was not. Scores are not directly comparable.

Another open-source project copies the same three question types and trains for probability output. Author-measured: 33 ms single question, 7.2 ms per question in batches. Model is available for download.

7. Risks and limitations

First: week one is prototypes and trial demos, almost no production. Treat the numbers as a tracking baseline. Prototypes and trial demos are 62.4% of 2,153 hands-on cases; production plus migrated cases are 3.7% (Figure 15). Seven days is too short to know whether prototypes stick.

Figure 15 Stage reached by hands-on cases

Source: FUNDA

Second: criticism clusters on hype, moat, accuracy and vendor self-testing. All four point to missing independent checks on capability and method. Across all discussion, top complaints are overheated promotion (7.0%) and “retrained open model, thin moat” (3.8%). In the same set, 40.6% of cases contain praise, mostly for speed and low cost (Figure 16).

")

Figure 16 Praise and criticism themes (share of all 6,277 cases; a case can have several)

Source: FUNDA

One skeptical write-up argues Jev is not a substitute for general LLMs: at most 255 preset options, no code or natural-language generation, no shown reasoning. As the author puts it, “jev is just a really smart switch statement”

Third: criticism is about positioning and method. It has almost never led to abandonment. Criticism appears in 25.1% of all discussion, but only 1 of 2,153 hands-on cases ended with Jev dropped after a trial. One week is too short to judge abandonment.

Fourth: among cases with a known outcome, failures are limited. Main causes are low accuracy and unsuitable tasks, not the speed or output format the vendor advertises. Of 1,581 cases with a known outcome, 14.2% did not succeed (partial, failed, or claimed but doubtful). Main causes: low accuracy (23.1%) and tasks outside Jev’s scope (13.8%); latency worse than claimed is 4.0% (Figure 17).

Figure 17 Outcomes of hands-on cases, and why cases failed

Source: FUNDA

Fifth: sample and automatic coding both carry systematic error. Every share is approximate. Of 6,277 counted cases,

  1. 7% involve KOLs or the vendor, 2.6% are vendor or promotional content, 11.0% come from project lists, and the 10 most

active authors account for 5.0%. Cases were classified by AI models. Two methods on the same 80 short comments agreed

  1. 0% on record type, 82.5% on main category, 83.8% on sentiment and 83.8% on audience, so sentiment, audience and use

shares carry error of similar size.

8. Conclusions

Jev’s opening is fast, low-value decisions inside existing systems where the answer comes from a preset list. It is not competing for text-generation work. Of 1,284 cases with a clear use: classification and routing 27.0%, real-time and games 18.8%, agent step decisions 16.1%. Of 2,153 hands-on cases, 30.9% are code projects developers published. Vendor claims of 444.6× cheaper, 193.6× faster and zero output-format errors drew attention. Users report median gains of 10× speed and 54.5× cost. Of 432 cases that name a comparison model, 48.6% put Jev against a small or open model.

Still unproven: whether production use lasts, whether confidence scores can be trusted, and how much independent evaluation exists. Of 2,153 hands-on cases, only 80 (3.7%) are in production, 62.4% are prototypes or trial demos, 19.5% are independent evaluations. Of all 6,277 cases: 1.0% offer calibration evidence, 0.3% record a migration, 3.0% are open-source replica attempts. Among 1,581 cases with a known outcome, 14.2% did not succeed. With 82.5%–88.8% agreement between the two automatic classifiers on 80 short comments, every share here is a first-week baseline, not a settled result.

9. Metrics to track

Is trial turning into production use? Watch the production share, now 3.7%. First-week use was mostly trial: prototypes and trial demos 62.4% combined. A rising production share as the sample grows, and production spreading from single decision points to linked steps, would mean users are moving from trying Jev to relying on it.

Are outsiders checking the vendor’s numbers? Independent evaluation is 19.5% of hands-on cases; calibration evidence is

  1. 0% of all discussion. A clean positive signal: independent re-tests on out-of-distribution data that publish

confidence against actual accuracy, not a single accuracy headline.

Is negative talk becoming real churn? Track criticism share (25.1%), abandonment after trial (1 case), and open-source replicas (3.0%) together. The moat holds only if criticism falls, abandonment stays low and replicas stop growing. Table 7 has current values and the direction to watch.

[![Production 3.7%, independent evals 19.5%, cheaper-but-less-accurate 14.9%, vendor-figure repeats 20.7%, criticism 25.1%.]( "Production 3.7%, independent evals 19.5%, cheaper-but-less-accurate 14.9%, vendor-figure repeats 20.7%, criticism

  1. 1%.")

Disclaimer and Terms of Use

Analyst Independence Statement

The views expressed in this report accurately reflect the personal and independent views of the analyst(s) regarding the securities and issuers discussed herein. No part of the analyst’s compensation was, is, or will be directly or indirectly related to the specific recommendations or views expressed in this report. The Company adheres to strict compliance procedures to ensure the integrity, security, and legality of the data sources and information utilized.

About FUNDA

Consensus AI Technology Pte. Ltd. (the “Company” or “FUNDA”) is an independent research firm incorporated in Singapore. The Company is not affiliated with any securities dealer, investment bank, or asset management firm, nor is it registered as an investment advisor or broker-dealer in any jurisdiction. The Company does not have any investment banking or underwriting relationship with the companies covered in this report, and does not accept sponsorship or funding from publicly listed companies.

No Investment Advice and Report Neutrality

The contents of this report and the information contained herein are for informational purposes only and do not constitute investment advice or a basis for business decisions. The views expressed herein represent solely the personal opinions of the respective authors at a specific point in time and do not reflect the position of FUNDA. As an independently operated platform, FUNDA maintains a neutral third-party status and makes no express or implied warranties regarding the accuracy, completeness, or applicability of such views.

No Offer or Solicitation: This report does not constitute an offer, solicitation, recommendation, or advice to buy, sell, or hold any securities or financial instruments, nor does it provide any legal, tax, or accounting advice.

Third-Party Data: All third-party data cited in this report is derived from public or authorized sources. The Company makes no representations or warranties, express or implied, as to the accuracy or completeness thereof.

Non-Participation in Decision-Making: The Company does not participate, directly or indirectly, in the investment decision-making process of its clients or readers. Readers should make independent investment decisions based on their own investment objectives and financial circumstances, and consult professional financial or legal advisors prior to executing any transactions.

Timeliness Statement

Given the rapid changes in market environments and technological trends, this report reflects market conditions as of the date prior to its publication. Relevant information may become outdated over time. FUNDA has no obligation to update the contents of this report in real-time.

Intellectual Property, Confidentiality, and Restrictions on Commercial Use

All reports, data, models, and content provided by the Company constitute its exclusive proprietary intellectual property and are protected by copyright and intellectual property laws.

Ownership of Rights: Unless otherwise expressly stipulated in a written agreement, all intellectual property rights (including copyrights) arising from or relating to this report are exclusively owned by the Company.

Authorized Use: Institutional clients may use this report internally within the scope permitted by their internal compliance policies; however, disclosure or sharing with any third party is strictly prohibited.

Prohibition on Commercial Use: Without the prior express written consent of the Company, no individual or entity may reproduce, republish, distribute, resell, repackage, or otherwise use this report or any part thereof for any commercial or profit-making purpose.

Legal Recourse: The Company reserves the right to pursue all available legal remedies and claims against any unauthorized commercial use, redistribution, or infringement of its intellectual property rights.

Forward-Looking Statements and Disclaimer

This report may contain forward-looking statements, including forecasts, estimates, and analyses of industry trends. Such statements are subject to risks and uncertainties that may cause actual results to differ materially from those anticipated. The Company shall not be liable for any direct, indirect, or consequential losses arising from the use of or reliance on the contents of this report.