2026.09.24 | 9K | ASIC | ASIC Roadmap Check II

音频信息

音频标题: 260924 - Merit ASIC技术路线调研II

相关标的：undefined

议题：

完整音频链接：

音频纪要总结

AI加速芯片、快速推理与异构计算专家访谈纪要

1. GPU与ASIC技术路线

1. GPU与ASIC的核心差异及优缺点是什么？

• GPU的主要优势

1. 适用于广泛工作负载，通用性和灵活性较强。

1. 能够支持训练、预填充、推理等多种任务，软件生态相对成熟。

1. 对模型架构变化和新模型的适应速度更快，部署门槛较低。

• ASIC的主要优势

1. 针对特定工作负载进行高度定制，能够在相同成本或功耗下提供更高性能。

1. 在解码、低延迟推理等特定任务中，可能明显优于通用GPU。

1. 每瓦性能和总体拥有成本具有潜在优势。

• ASIC的主要局限

1. 灵活性较低，适用工作负载范围相对有限。

1. 模型架构发生变化时，硬件和软件适配成本可能更高。

1. 软件生态、人才供给及模型覆盖范围通常弱于CUDA。

• GPU与ASIC之间的核心权衡，是通用性与特定任务执行效率之间的权衡。

2. 未来是否会出现不同ASIC分别执行不同AI任务的趋势？

• 行业正在走向“异构推理”，即由GPU和不同类型ASIC分别执行最适合的任务。

• 典型任务拆分包括：

1. GPU负责预填充等适合高并行计算的任务。

1. LPU等专用芯片负责解码和低延迟任务。

1. 晶圆级产品承担快速推理、大模型映射等工作。

• 以英伟达为例，其GPU在预填充环节具备优势，而Groq类专用架构在解码环节更有竞争力；二者结合可以形成更完整的推理方案。

• AWS与Cerebras的合作也体现了类似思路，即将专用推理硬件接入更大的云计算生态。

• 未来预计会有更多公司采用GPU与ASIC混合部署模式，前沿模型公司也可能自行开发专用芯片。

• OpenAI的Jalapeno目前主要面向推理，未来可能进一步演进，但具体路线仍存在不确定性。

2. AI芯片性能评价体系

1. 应如何比较不同GPU和ASIC？

• 比较不同硬件时，首先要确保提供的是相同用户体验，否则单纯比较峰值性能意义有限。

• 主要评价指标包括：

1. 每秒生成Token数：衡量单个请求或单个用户感知到的生成速度。

1. 每瓦每秒生成Token数：衡量能效，是判断不同芯片产品竞争力的重要指标。

1. 总体吞吐量：衡量单位时间内系统面向全部用户生成的Token总量。

1. TCO（总体拥有成本）：包含电力、硬件、部署及运维成本。

1. 首Token生成时间：反映用户发出请求后获得首次响应的延迟。

1. 模型部署时间：反映平台支持新模型的速度和企业采用门槛。

1. 模型覆盖范围：衡量平台可以支持多少种模型及模型架构。

• 受访者认为，电力成本约占运营成本的50%，因此每瓦性能和TCO的重要性非常高。

• 即使某一系统峰值性能很高，如果功耗、部署和运维成本过高，也未必具备商业优势。

2. 不同用户体验会如何影响硬件比较结果？

• 高频、复杂请求用户更重视响应速度、延迟和大上下文能力。

• 普通用户未必能感知每秒数百Token与每秒数千Token之间的差异，更关注稳定性和成本。

• 企业级用户还会关注模型兼容性、部署速度、IT支持和运维复杂度。

• 因此，硬件比较应限定在相同模型、相同精度、相同上下文长度和相同服务质量下进行。

3. 每秒Token速度和总体吞吐量有什么区别？

• 每秒Token速度

1. 主要衡量单次计算或单个用户请求的响应速度。

1. 用户可以直接感知。

1. 对聊天机器人、编程助手、实时语音和智能体等应用更重要。

• 总体吞吐量

1. 指系统在单位时间内面向所有用户处理和生成的Token总量。

1. 更接近数据中心整体容量和运营效率。

1. 可通过批处理、并发调度等方式提升，单个用户通常无法直接感知。

• 两者存在一定权衡：优化单用户低延迟不一定能够同时实现最高系统吞吐量。

3. 不同AI应用的核心性能需求

1. 聊天机器人场景最关注哪些指标？

• ChatGPT类聊天机器人最关注：

1. 首Token生成时间。

1. 用户交互延迟。

1. 单用户Token生成速度。

• Groq、Cerebras等快速推理平台的价值，主要体现在降低交互延迟、提升实时响应能力。

2. 大规模文档处理最关注哪些指标？

• 大规模文档处理对单次请求延迟的敏感度相对较低。

• 核心指标包括：

1. 总体吞吐量。

1. 每Token成本。

1. 每瓦性能。

1. 数据中心总体TCO。

• 在此类批量处理场景中，系统容量和经济性通常比单用户速度更加重要。

3. 实时视频、语音生成场景最关注哪些指标？

• 视频生成会出现阶段性的Token吞吐峰值，因此峰值Token吞吐量非常重要。

• 实时语音和视频应用同时关注：

1. 低延迟。

1. 峰值吞吐能力。

1. 响应稳定性。

• 如果系统不能稳定处理峰值负载，即使平均性能较高，也难以满足实时业务需求。

4. 训练场景最关注哪些指标？

• 训练场景主要关注FLOPS及有效计算利用率。

• 同时也需要综合考虑：

1. 芯片间互连效率。

1. 内存容量和带宽。

1. 集群扩展能力。

1. 软件生态及训练框架适配能力。

5. AI编程场景最关注哪些指标？

• AI编程的核心指标包括：

1. 低延迟：用户需要快速获得代码建议和修改反馈。

1. 大上下文窗口：需要保留代码、错误信息、测试结果和多轮交互内容。

1. 大容量内存：支持200K乃至100万Token级别的上下文。

1. 推理质量：模型生成代码的准确性和解决问题的能力。

• 用户编程时往往需要反复进行“生成代码—本地测试—反馈错误—继续修改”的循环。

• 随着代码、日志、工具输出和历史对话不断累积，上下文会快速变长，因此AI编程对大上下文和低延迟都有较高要求。

4. 快速推理市场空间与需求结构

1. 快速推理市场未来有多大？

• 快速推理市场规模较难精确预测，但受访者预计，未来5年可能形成约500亿美元的市场。

• 主要增长驱动力包括：

1. AI编程。

1. 实时智能体。

1. 企业级智能体自动化。

1. 实时语音和视频应用。

• Meta推出Muse智能体后的市场反馈，被视为实时智能体需求快速增长的案例。

2. 用户是否真的需要每秒2,000个Token的超高速推理？

• Cerebras和Groq声称可以实现每位用户每秒2,000个以上Token，但实际运行速度通常约为每秒400至500个Token。

• 对多数人机交互场景而言，每秒400至500个Token已经明显高于实际需求。

• 达到一定速度后，用户难以继续感知性能差异，成本和稳定性会变得更加重要。

• 每秒1,000至2,000个Token更适用于：

1. 持续循环运行的智能体。

1. 企业级自动编程任务。

1. 夜间长时间运行、需要在次日完成任务的智能体。

1. 高实时性语音智能体。

• 对大部分推理市场而言，当前阶段总体吞吐量的重要性高于极致单用户速度。

5. Cerebras的竞争力、发展前景与风险

1. Cerebras未来是否有望成为规模较大的商业化公司？

• Cerebras面临英伟达、Groq及其他推理ASIC厂商的竞争，但市场普遍看好其发展前景。

• 其核心产品采用整片晶圆，而非将晶圆切割成独立裸片后再进行封装。

• 这一晶圆级架构在芯片物理设计和制造方式上具有较强创新性，短期内不容易被复制。

• 受访者预计，其技术壁垒可能维持约3至4年。

• Cerebras仍需进一步证明：

1. 能够持续获得更多客户。

1. 具备稳定的模型效率优势。

1. 能够扩大支持的模型范围。

1. 能够形成更完善的软件和开发者生态。

2. Cerebras的核心技术壁垒是什么？

• 核心壁垒是晶圆级集成和制造路线，而非单纯依靠专利。

• 传统芯片通常将晶圆切割成多个裸片，再通过Chiplet、封装或电路板互连。

• Cerebras直接使用整片晶圆，使计算单元在晶圆上完成连接，有利于降低通信延迟、减少外部互连开销。

• 其与多家公司的合作说明该技术已经从实验室阶段进入工业化阶段，具备一定的可生产性和扩展性。

• 受访者认为，Cerebras最重要的优势是率先实现晶圆级技术的商业落地。

3. Cerebras与OpenAI Jalapeno短期和长期是什么关系？

• 短期关系

1. OpenAI仍高度依赖外部推理算力。

1. Cerebras能够提供快速推理能力，因此短期合作基础仍然存在。

• 长期关系

1. 随着Jalapeno及后续自研芯片发展，OpenAI与Cerebras可能形成竞争。

1. OpenAI拥有较强的芯片开发能力和模型软硬件协同优势。

1. Cerebras在晶圆级架构方面仍具备OpenAI短期难以复制的护城河。

• 双方未来可能同时存在合作与竞争，Cerebras仍面临被客户自研芯片替代部分需求的风险。

4. 哪些模型架构变化会影响Cerebras的优势？

• 正面因素

1. 模型规模持续扩大。

1. 上下文窗口继续增长。

1. 内存和数据搬运需求增加。

• 负面因素

1. 模型蒸馏降低推理所需内存。

1. MOE架构仅激活少数专家，使模型总参数增长但单次推理计算量和数据传输量未同步增长。

1. 每个Token传输的字节数趋于稳定。

• 对Cerebras而言，关键指标不是模型总参数量，而是每个Token实际需要传输的字节数。

• 如果模型规模增加，但单Token数据传输量因MOE、蒸馏等技术保持平稳，Cerebras的大容量内存和晶圆级优势可能弱化。

5. Cerebras会成为独立平台还是异构系统中的组件？

• 受访者认为，Cerebras最终有可能像Groq一样，成为大型异构计算生态中的一个专业组件。

• AWS合作有利于Cerebras扩大客户覆盖和商业落地，但也可能使其被定位为云平台中的特定推理模块。

• 仅依靠晶圆级技术，不一定能够长期支撑独立平台地位。

• 若要长期保持独立性，Cerebras需要：

1. 扩大软件生态。

1. 提升模型覆盖范围。

1. 实现客户和业务多元化。

1. 与更多云厂商、GPU厂商及ASIC厂商建立合作。

• 除英伟达外，AMD、AWS、Intel及其他硬件厂商均可能与Cerebras开展合作。

• Cerebras要继续增长，需要尽可能扩大合作伙伴网络。

6. CUDA生态与其他软件栈的差距

1. CUDA、AWS和Cerebras的软件生态如何比较？

• 受访者给出的总体排序为：

1. CUDA

1. AWS软件栈

1. Cerebras软件栈

• CUDA经过十余年迭代，拥有庞大的客户群、开发者群体和问题解决经验。

• AWS在模型覆盖范围和软件解决方案完整度上优于Cerebras。

• Cerebras对于已经支持的简单模型较容易使用，但：

1. 支持的模型范围有限。

1. 熟悉其软件栈的工程师较少。

1. 新问题缺乏成熟解决方案。

• 英伟达的核心护城河不只是芯片性能，更是CUDA形成的完整软件生态。

2. 为什么终端用户更倾向于采用CUDA？

• CUDA支持新模型的速度可能低至1天以内，通常约为2至3天。

• Neuron、Cerebras等平台支持新模型可能需要数天至数周。

• CUDA的用户基数较大，常见错误通常已经有成熟解决方案。

• 在Cerebras等新平台上遇到的问题可能是首次出现，需要用户、平台和IT团队反复沟通调试。

• 人才招聘方面，市场上熟悉CUDA的工程师较多，而熟悉AWS Neuron或Cerebras软件栈的人才相对稀缺。

• 企业选择CUDA可以降低部署、调试、招聘和运维风险。

3. CUDA在前沿模型训练中的份额有多高？

• 受访者估计，目前约90%的模型在CUDA上训练。

• 过去这一比例可能曾达到95%以上，但随着推理需求增长和其他硬件平台进入，份额有所下降。

• OpenAI的模型主要在CUDA上训练，同时似乎也开始采用AMD。

• Anthropic的训练体系可能有所不同，但受访者因相关限制未进一步评论。

• CUDA目前仍在训练市场占据主导地位，但企业开始认识到，并非所有模型和工作负载都必须使用同一种通用硬件。

4. 将新模型部署到非CUDA平台需要多长时间和多少人？

• 将新模型部署到AMD、Cerebras、AWS等非CUDA平台，通常至少需要约4周。

• 主要工作包括：

1. 内核适配。

1. 编译器适配。

1. 模型支持审批。

1. 性能调优和微调。

• 一般可按约5名工程师估算，包括：

1. 2至3名资深内核或编译器工程师。

1. 若干初级工程师。

• 不同平台的相对部署速度：

1. AMD可能最快。

1. Cerebras其次。

1. AWS Trainium可能最慢。

• AMD和Cerebras通常需要2至4周或4至6周；若Cerebras尚未支持对应模型，周期会更长。

7. 英伟达、OpenAI及新兴ASIC厂商的竞争格局

1. 英伟达引入Groq团队和技术会产生什么影响？

• 受访者认为，这项交易或人才收购对英伟达非常有利。

• 英伟达此前的主要短板是推理解码延迟，而Groq类架构可以补足这一能力。

• 组合后，英伟达可以同时提供：

1. 领先的GPU预填充能力。

1. 更具竞争力的低延迟解码方案。

1. 成熟的CUDA软件生态。

1. 更快的部署速度。

• 英伟达同时拥有较强定价权，新进入者很难同时在价格、软件和部署速度上与其竞争。

• 该交易验证了GPU与ASIC混合部署路线，预计将推动更多企业采用异构推理方案。

• 如果整体AI市场继续高速增长，各类参与者仍可能共同受益；若市场停止增长，小型ASIC厂商将承受更大压力。

2. OpenAI Jalapeno会如何影响竞争格局？

• OpenAI首次开发芯片，据受访者了解，从规格定义到流片约用9个月，明显短于行业通常约18个月的周期。

• OpenAI借助AI工具提升了芯片设计效率，开发周期约为传统公司的二分之一。

• Jalapeno与Blackwell对比时，每瓦性能据称高出约50%。

• 鉴于电力成本约占运营成本的50%，每瓦性能提升可能显著改善TCO。

• 这一案例表明，前沿模型公司可能逐步开发自有芯片，原因包括：

1. 更了解自身模型和工作负载。

1. 能够进行软硬件协同优化。

1. 可利用智能体AI缩短芯片设计周期。

1. 能够降低对外部GPU和云厂商的依赖。

• 前沿模型公司的自研ASIC可能对英伟达、AWS及第三方推理芯片厂商形成长期竞争。

3. Anthropic是否会开发自有ASIC？

• Anthropic已聘请曾参与Google TPU项目的Amir Salek负责硬件业务，并正在组建团队。

• 虽然Anthropic尚未公开自研芯片计划，但这一人事安排被视为明确的战略信号。

• 受访者判断，Anthropic未来几乎确定会开发自有ASIC。

• 关于Anthropic与Broadcom约5吉瓦需求所对应的具体芯片类型，受访者因相关限制未作判断，但确认其规模属于大型数据中心级别。

4. MatX可能成为怎样的竞争者？

• MatX由Google TPU前核心架构师及多名Google员工创立。

• 公司重点研发推理芯片，强调总体吞吐量和定制化能力。

• 受访者认为，MatX未来规模可能达到Groq水平，甚至超过Groq。

• 其最终竞争力仍取决于芯片量产、软件生态、客户验证和商业化落地情况。

8. AI对芯片设计周期和产业瓶颈的影响

1. AI芯片设计周期未来会缩短到什么程度？

• 传统芯片从规格定义到流片通常需要12至18个月。

• OpenAI已经将这一阶段缩短至约9个月。

• 若加上芯片启动、验证和量产准备，当前完整周期仍可能约为1年半。

• 随着智能体AI工具成熟，受访者预计：

1. 从规格定义到流片的周期可能缩短至4至5个月。

1. 整体设计流程可能缩短至6个月以内。

• RTL和Verilog编码已经开始自动化，AI能够显著提升代码生成效率。

• 过去可能需要约20人参与的设计验证工作，未来可由AI智能体承担相当一部分，从而减少人员投入并加快验证速度。

2. 哪些环节将从芯片设计瓶颈转变为制造瓶颈？

• 软件驱动的设计、编码和验证环节将因AI而快速提效。

• 未来最大的限制可能转向晶圆制造端，包括：

1. 台积电、三星、英特尔能否及时提供流片产能。

1. 先进制程产能能否满足大量ASIC项目同时流片。

1. 封装、测试和量产资源能否同步扩张。

• 随着更多公司缩短芯片设计周期，大量项目可能在相近时间进入流片阶段，晶圆厂无法同时满足全部需求。

• 因此，先进制造产能和与代工厂的合作关系将成为重要竞争壁垒。

• 英伟达与台积电保持长期紧密合作，正是为了提前锁定先进制程和制造资源。

结构化总结表格

新版prompt提炼之后的转译

提问者：首先，请您分享一下对GPU和ASIC技术路线及架构的看法。不同GPU和ASIC分别有哪些优缺点？

回　答　者：总体而言，GPU适用于广泛的工作负载，而非特定工作负载。ASIC的定制化程度更高，灵活性相对较低，但在相同成本或功耗下能够实现更高的性能，主要面向特定的目标工作负载。因此，两者之间的核心权衡在于适用范围与执行性能。

提问者：未来是否会出现不同ASIC分别执行不同任务，或更适合不同工作负载的趋势？例如，谷歌的TPU分为推理TPU和训练TPU；英伟达收购了Groq，其LPU可能更适合解码；此外，Cerebras的晶圆级产品可能更适合快速推理和解码。您是否认为，不同ASIC将分别适用于不同类型的工作负载？

回　答　者：确实如此，这正是行业的发展趋势，通常称为异构推理。正如您所提到的，英伟达收购了Groq，Cerebras也在与AWS合作。

以英伟达为例，其GPU能够高效处理部分任务。例如，当用户提出问题时，系统首先会一次性将所有数据载入内存，这类任务GPU能够快速完成。但对于某些任务，GPU的处理能力不及Groq等公司的专用产品，例如用于解码的LPU。因此，英伟达可以将两者结合使用。通过对Groq的收购或人才收购，英伟达已引入其联合创始人和总裁。AWS也采取了类似做法。

回　答　者：未来可能会有越来越多的公司这样做。正如我之前提到的，其中存在权衡。我认为 OpenAI 可能会是下一个，但我并不确定。正如您提到的 Jalapeno，目前仅用于推理，但预计未来会继续发展，至少目前湾区业内人士普遍这样认为。

提问者：我们应该如何比较不同的 GPU 或 ASIC？是否有通用的基准、指标或规格，可用于比较不同的 GPU 和 ASIC？具体应该如何比较不同的 ASIC？

回　答　者：在提供相同用户体验的前提下，可以关注每秒能够生成多少个 Token，或者结合功耗考量，即每瓦每秒生成的 Token 数。此外，还需要关注吞吐量，以及运行该平台的总体拥有成本，即 TCO。

TCO 同样重要，实际上占运营成本的 50%。一个系统可以具备非常高的性能，但每瓦性能或 TCO 才是主要衡量指标。目前采用的一项标准就是每瓦每秒生成的 Token 数，这一指标基本可以用于判断不同产品的优劣。虽然还有其他标准，但这是主要标准之一。

比较时还应基于相同的用户体验。不同用户的使用方式有所不同，例如，一些用户的需求较为激进，可能会非常快速地发出请求；另一些普通用户并不使用这类系统创建应用。因此，比较时应确保面向相同类型的用户体验。这是目前业内采用的基本基准之一。

此外，还需要考量部署速度。例如，Cerebras 拥有大型晶圆，但能否快速在其上部署新模型也很重要。因此，部署所需时间也是一项标准或基准。

提问者：不同场景、不同应用和不同类型的工作负载，通常分别需要关注哪些特性或指标？在哪些场景中，每位用户每秒生成的 Token 数这一速度指标最为重要？

提问者：在不同场景中，哪些指标最为重要？正如您提到的，部署时间非常关键。您能否具体介绍一下？

回　答　者：以ChatGPT这类聊天机器人为例，首个Token生成时间非常重要，因此延迟是关键指标，需要关注用户每秒交互时的延迟。这正是Groq或Cerebras所处领域的重要性所在。

另一个指标是吞吐量。如果需要处理大量文档，延迟可能并不重要，但每Token成本较为关键，因为需要处理大量数据，而不取决于用户侧的使用延迟。

此外，部署时间也很重要。当初创企业或其他公司需要采用Cerebras、Groq等新公司的平台时，能否在新平台上运行其模型就非常关键，这属于企业级层面的考量。

在视频等应用中，例如要求大语言模型生成视频时，峰值Token吞吐量非常重要。与动态功率存在峰值曲线类似，如果绘制Token吞吐量随时间变化的曲线，也会看到峰值。使用大语言模型生成视频时，Token吞吐量会达到峰值，因此这一指标对于实时视频或语音应用非常重要。

在训练场景中，主要关注FLOPS指标。

提问者：明白了。那么，AI编程场景如何？此外，我们注意到，同一个客户或同一家公司，例如Cognition，既会向Nevius购买云服务，也会向Cvas购买云服务。Cognition可能是AI编程或类似领域的参与者。为什么Cognition会向不同的服务提供商购买云服务？

回　答　者：您提到AI编程，主要是想了解这一场景中哪些指标最重要，对吗？

提问者：是的，这方面的信息也会有所帮助。

回　答　者：AI编程场景主要关注延迟。因为用户编程时需要快速获得响应，所以更强调交互延迟。此外，较大的上下文也需要大容量内存。

回　答　者：由于人们往往并不清楚自己具体想编写什么代码，因此会反复调整。在这种情况下，模型的记忆能力和上下文窗口大小就非常重要，例如能够容纳200k还是1,000,000个Token。此外，推理质量本身也是一项重要的评估基准，对用户编程同样重要。

提问者：您提到人们可能并不清楚自己想编写什么代码，应如何理解？

回　答　者：以芯片设计人员为例，他们首先会要求大语言模型编写一段代码。模型生成代码后，可能会询问用户是否希望采用其他方式。用户从AI获得代码后，会在自己的系统中进行测试。例如，芯片设计人员可能使用Innovus或编写Verilog。发现错误后，他们会继续与模型沟通，复制并粘贴相关内容，检查AI模型生成的输出。

因此，随着AI生成内容与工具处理结果之间不断进行复制、粘贴和反复交互，上下文通常会变得很长。这就是编程场景需要较大上下文容量的原因，因为用户与AI聊天机器人之间会发生大量反复的信息传递。

提问者：在这种情况下，延迟也很重要，对吗？

回　答　者：是的，这种场景需要较低的延迟，低延迟非常重要。

提问者：在快速推理领域，目前已有Cerebras等快速推理ASIC厂商，也有一些规模较小的初创企业。快速推理当前及未来的市场规模或市场潜力如何？快速推理未来可能占整体推理市场的多大比例？

回　答　者：这一问题很难准确判断，但从增长趋势来看，快速推理可能会形成一个相当可观的市场，例如在5年内达到500亿美元。不过，这也取决于整体市场规模。目前可以确定的是，这一市场增长非常迅速，尤其受到AI编程和实时智能体部署的推动。

Meta推出Muse的情况表明，实时智能体存在巨大需求。Meta刚刚推出这款Muse智能体，其下载量在1周内就超过了OpenAI。

提问者：Cerebras和Groq声称，每位用户的Token生成速度可达到每秒2,000个以上，即每位用户每秒2,000个Token。对于一般的AI编程场景，可能每位用户每秒300个或500个Token就已经足够。您能否估算一下，超高速推理在整体推理市场中的占比？具体而言，每位用户每秒需要生成1,000个以上Token的需求占比是多少？换言之，尽管Groq和Cerebras声称可以达到每秒2,000个Token，但用户通常实际需要多快的速度？

回　答　者：虽然它们宣称可以达到每秒2,000个Token，但目前实际速度通常略低，一般只有每秒400个或500个Token。我认为，这已经远高于实际需求。达到一定速度后，用户无法感知其中的差异，因此成本更低可能更加重要。

速度主要对持续循环运行的智能体较为重要，在企业层面尤其如此。每秒2,000个Token的速度在企业生产环境中非常重要，例如公司员工使用智能体AI解决方案进行编程，或者让Google的Jetski等智能体循环运行一整夜，并希望次日早上回来时任务已经完成。在这类场景中，速度非常重要。

但对于大部分推理市场，我认为吞吐量比速度更重要。除非是实时智能体，例如需要快速响应的语音智能体。就现阶段而言，吞吐量的重要性高于速度。

提问者：吞吐量与每位用户每秒生成的Token数之间，主要区别是什么？

回　答　者：吞吐量是指大语言模型进行计算时生成的Token总量。

回　答　者：每单位时间内处理的任务量涉及2个方面。用户输入内容并获得响应时，后端会执行相应计算。“每秒生成的Token数”主要指计算过程的速度，而吞吐量是指面向所有用户总体生成的Token数量，也可以理解为系统容量。用户能够直接感知前者，但通常不会感知后者，因为系统可以通过批处理等方式并行处理请求，两者之间存在一定的权衡。

提问者：您如何看待Cerebras未来的发展潜力和增长前景？您认为它未来会成为规模更大、商业上更成功的公司吗？

回　答　者：Cerebras面临来自NVIDIA和Groq的竞争，但总体来看，投资界普遍认为其发展表现会比较好，而且它的解决方案并不容易复制。Cerebras直接使用整片晶圆，并未将其切割成独立裸片。可以将其理解为Chiplet，但从芯片物理设计的角度看，我认为这是极具创新性的做法：它使用整片晶圆，裸片在晶圆上已经彼此连接。因此，其他公司很难轻易复制，其竞争壁垒或防御能力至少可以维持3、4年。

当然，Cerebras仍需获得更多客户，并证明其模型具备效率优势。但从华尔街的观点以及湾区业内人士的讨论来看，其发展前景非常好。我认为Cerebras还与OpenAI达成了某种合作。不过，我们也需要考虑到OpenAI目前正在开发Jalapeno，因此双方是否还会继续合作，以及Cerebras是否会采取类似NVIDIA对Groq所采取的做法，仍有待观察。由此来看，Cerebras也存在一定的看空情形。

提问者：Cerebras与OpenAI达成了一项规模很大的合作，而OpenAI刚刚发布了Jalapeno。Cerebras的WSE与OpenAI的Jalapeno这2类ASIC之间会是什么关系？它们是否会相互竞争？

提问者：未来，这两类ASIC之间是否存在合作空间？

回　答　者：我认为这取决于短期和长期情况。OpenAI目前仍高度依赖推理能力，而这正是Cerebras所提供的。因此，短期内我认为不会有问题。但长期来看，双方可能会形成竞争。OpenAI或许仍会与Cerebras合作，具体情况还有待观察。Cerebras未来存在受到冲击的可能。

另一方面，OpenAI的创新能力很强，能够以更快的速度开发芯片。不过，Cerebras的晶圆级解决方案本身具备一定的护城河。OpenAI至少在晶圆层面可能无法独立实现Cerebras已经完成的技术，因此最终结果仍有待观察。

提问者：从软件生态系统的角度来看，Cerebras与CUDA相比表现如何？此外，Google的TPU和AWS等其他参与者与CUDA相比又如何？

回　答　者：总体而言，我认为CUDA是最好的。CUDA已经发展了超过10、20年，也可能是15年，拥有大量使用数据，许多客户已经采用，并经历了多次迭代。

由于协议限制，我无法评价TPU，但可以谈谈AWS。AWS的软件解决方案优于Cerebras。在模型覆盖范围方面，AWS的排名高于Cerebras。Cerebras比较适合非常简单的模型，或者说，对于其支持的模型，使用起来较为容易，但其覆盖范围很有限，而且熟悉这套软件栈的人并不多。

因此，在软件解决方案方面，我认为CUDA排名第1，其次是AWS，然后是Cerebras。CUDA基本可以即插即用，能够快速运行用户所需的内容。这也是AMD至今未能追上Nvidia的原因。问题从来不在于芯片，而在于Nvidia所提供的软件解决方案。即使AMD的芯片在功耗等方面表现更好，甚至芯片布局也更优，软件生态仍然是关键。

回　答　者：但其他厂商仍然无法赶上英伟达，主要原因在于 CUDA。我认为，这一护城河仍将由英伟达保持。英伟达在这一领域拥有大量人才，也投入了大量资金。正因如此，一些公司选择通过收购来构建自己的软件栈，我认为 AMD 也收购了相关公司。Cerebras 在这方面还有大量工作要做。关于 TPU，我无法讨论。

提问者：从终端用户的角度来看，CUDA 与其他厂商的软件栈有何区别？从开发者的角度看，CUDA 的生态系统更加成熟和完善，而且已经迭代多年。对于终端用户而言，这种差异是否体现在新模型的支持速度上？例如，CUDA 可以在第0天支持新模型，而其他厂商可能需要1周或1个月。具体差异是什么？

回　答　者：以 CUDA 为例，支持新模型可能需要2天、2至3天，甚至不到1天；而 Neuron 和 Cerebras 可能需要数天至数周。

从用户角度看，如果出现无法正常运行的情况，还需要进行调试并联系 IT 团队，期间会有大量反复沟通和处理流程，因此需要较长时间。这也是用户更倾向于选择 CUDA 的原因。

如果发生错误，CUDA 可能已经有相应的解决方案，因为此前已有其他用户遇到过同类问题。但在使用 Cerebras 时，该问题可能是首次出现，用户就可能因此陷入困境。

从公司的角度看，招聘也是一项考量因素。如果需要招聘相关工程师，公司可以找到熟悉英伟达 CUDA 的人才，但很难招聘到熟悉亚马逊或 Cerebras 解决方案的人才，这也是一个问题。

此外，许多问题此前已经在 CUDA 上得到解决。对于正在处理的问题，用户可能已经能够找到现成的解决方法。综合来看，这些因素都很重要。

提问者：所有前沿模型是否都在CUDA生态系统，也就是CUDA软件栈中训练？我的理解是否正确？

回　答　者：您的理解基本正确。我认为大约90%的模型是在CUDA上训练的。OpenAI的模型是在CUDA上训练的，Anthropic可能不是，我认为Anthropic的模型并非在CUDA上训练。OpenAI目前似乎也开始采用AMD。此前这一比例超过90%，一度达到95%，但现在可能为90%或更低，主要是因为推理需求增长。

CUDA过去一直是首选，目前仍占据主导地位，但其他公司正在逐步追赶，原因同样是每瓦性能。企业逐渐意识到，并不需要为所有模型采用同一种通用解决方案。

例如MatX，这家公司刚刚完成融资，目前估值达到10 billion美元。他们正在研发推理芯片，并声称其产品是性能最好的推理芯片，最终效果仍有待观察。

提问者：您说的是哪家公司？

回　答　者：MatX。可以研究一下MatX。我认为其未来规模可能会达到Groq的水平，甚至超过Groq。其创始团队来自Google，创始人是Google TPU的主要架构师。离开Google并创办MatX时，他还带走了许多其他Google员工。该公司正在开展大量定制化工作，重点提升吞吐量，目标是打造性能最好的推理芯片。

回到您的问题，目前CUDA的占比约为90%。

提问者：也就是说，目前在训练领域，CUDA仍占据主导地位，占比可能为90%，对吗？

回　答　者：是的。

提问者：对于其他厂商而言，如果要支持一个在CUDA生态系统中训练的新前沿模型，通常需要多长时间？以AMD为例，需要多少天，以及投入多少人力和工程师？对于Google、AWS及其他ASIC厂商，情况又如何？

提问者：在 CUDA 上进行训练需要多少时间和多少名工程师？在 AMD 上又需要多少时间和多少名工程师？

回　答　者：这些实验室通常每周或每2周迭代1次。在 AMD 及其他平台上，训练基本可能需要2个月，但 CUDA 可能会快得多。

提问者：我想确认一下，我问的不是训练时间，而是部署新模型所需的时间。在 CUDA 生态系统中，新模型可能在1天内完成部署；但如果将新模型部署在 AWS Trainium、Google TPU 或 Cerebras 系统上，需要多长时间和多少名工程师？

回　答　者：我明白了。部署可能需要4至5周，主要工作涉及内核适配，之后还需要获得模型支持方面的批准，并进行微调。因此，我认为至少需要大约4周。

工程师方面，最多可能需要5名具备相关知识的人员，其中可能包括2名或3名资深人员以及初级人员，主要需要内核工程师和编译器工程师。总体可以按5名工程师估算。

提问者：AMD、AWS、Google 和 Cerebras 所需的部署时间是否会有所不同？例如，AMD 可能需要1周，而 Cerebras 可能需要4周。不同平台之间的部署时间是否存在差异？

回　答　者：所需时间大致相同。AMD 和 Cerebras 的情况较为相似，因为它们与 CUDA 非常接近。Cerebras 需要先支持该模型；如果已经支持，所需时间可能相同，大约为2至4周，也可能是4至6周。我认为至少需要4周，因为引入新模型后必须完成相应流程。据我了解，AWS 所需时间更长，但 Google TPU 的情况我无法评论。

回　答　者：AMD可能是速度最快的，其次是Cerebras，AWS Trainium最慢。

提问者：英伟达通过人才收购方式收购Groq，可能会产生哪些影响？这对GPU和ASIC市场的竞争格局意味着什么？

回　答　者：我认为这是一次非常好的收购，会影响很多公司。英伟达此前面临的主要问题是延迟，而此次收购解决了这一问题。英伟达的GPU本身没有问题，其预填充GPU解决方案一直处于领先地位。现在，英伟达拥有了一款极具竞争力的产品，既具备领先的预填充能力，又采用了领先的解码方案。

英伟达也拥有定价权。其他新进入者即使希望采用相同方式，也很难在价格上与英伟达竞争，而且英伟达的部署速度也很快。

这次收购也说明混合使用GPU和ASIC的路线已经得到验证。英伟达的做法将推动更多企业混合使用GPU和ASIC。我认为AWS也在尝试与Cerebras采取类似的做法。由于目前采用这种方式的公司并不多，因此整体而言，这会对行业有所帮助，但也可能带来冲击。

只要整体AI市场持续增长，就会使所有参与者受益。如果市场规模保持不变、不再增长，那么规模较小的参与者将会受到影响。

提问者：OpenAI的Jalapeno会对当前竞争格局产生什么影响？

回　答　者：我认识参与开发这款芯片的人员，他们都非常优秀，其中一些人来自Google。他们利用自己的AI技术，在其他公司一半的时间内完成了芯片开发。这意味着，如果芯片出现问题，他们可以再次进行开发。

OpenAI是首次开发芯片，但所用时间仅为其他公司的一半。通常，从规格定义到流片的芯片开发周期为18个月，而OpenAI大约用了9个月，仅为一半时间。

他们将这款芯片与Blackwell进行了比较，结果显示，其每瓦性能比英伟达高出50%。功耗与成本相关。

回　答　者：评估TCO，即总运营成本时，电力成本占50%。如果每瓦性能提升50%，基本意味着运营成本也会相应改善50%。因此，我认为这是一个非常明显的信号：这些前沿模型公司最终都会开发自有芯片。它们可以比硬件公司做得更好，因为现在公司内部拥有智能体AI解决方案，能够加快芯片设计周期。

Recursive Intelligence所做的就是类似的事情。他们声称可以在1天内完成整个芯片的布局规划和布置，而过去需要1周。因此，我认为这些公司将对NVIDIA形成强有力的竞争，甚至也可能对Amazon等其他公司构成竞争。

提问者：Anthropic目前是否也在开发自己的ASIC？

回　答　者：是的。Anthropic刚刚聘请Amir Salek负责硬件业务。他也曾任职于Google，是Google TPU项目的主要负责人之一。虽然他不是该项目的创始人，但属于核心人员。Anthropic刚刚聘请了他，目前他正在组建团队。

Anthropic现阶段采用租赁方式，虽然尚未公开相关计划，但聘请了一位此前有过相关经验的人，意味着公司将推进这项工作。因此，我认为Anthropic 100%会开发自己的ASIC。

提问者：我们从Broadcom的财报电话会获悉，Anthropic与Broadcom签署了一笔规模很大的交易。Broadcom的Hawk 10表示，Anthropic已与Broadcom签署了约5吉瓦的需求。您认为这5吉瓦需求对应的是Anthropic自己的ASIC，还是Google TPU？我们知道Anthropic可能在使用Google TPU。

回　答　者：由于我是TPU团队的成员，无法评论Google TPU。但可以确认，这一庞大的数字属于数据中心规模，规模非常大。

提问者：关键来源和核心竞争优势有哪些？

提问者：从专利、IP、专有技术、编译器技术和客户关系等方面来看，您认为 Cerebras 的核心优势是什么？

回　答　者：我认为，Cerebras 的核心优势在于其开创的晶圆级技术路线及相应的开发方式。芯片制造通常是在一整块晶圆上完成，再将晶圆切割成多个裸片，用这些裸片构建 Chiplet，或者将其组装到封装或电路板上。

在单个裸片内部，所有组件都通过金属层连接，因此可以实现非常低的延迟。但采用 Chiplet 时，芯片彼此分离，Chiplet 之间需要通过互连进行通信，互连介质可能是铜，目前也开始采用光。

Cerebras 的优势在于创新了新的制造方式，将所有组件集成在单块晶圆上，并直接使用整块晶圆。这是其最主要的技术壁垒。基于这一技术，Cerebras 能够映射大量模型，也能适配不同规模的模型，并进行大规模部署。

Cerebras 与多家公司达成的合作证明，这并非停留在高校技术或高校项目阶段，而是已经达到工业级水平，具备可扩展性和可生产性。专利并不是其最主要的优势，率先进入市场才是关键。Cerebras 在其他公司之前实现了这一技术，这正是其优势所在。

提问者：AI 模型架构发生哪些变化，可能会对 Cerebras 基于晶圆级 SRAM 的系统产生正面或负面影响？

回　答　者：如果模型规模持续扩大，特别是当前行业正在向更长上下文发展，这将进一步凸显 Cerebras 的优势。但未来也可能出现内存不再那么重要的情况，例如行业更多采用模型蒸馏。如果行业朝着降低内存需求的方向发展，Cerebras 相关优势的重要性可能会下降。

回　答　者：基本上会下降。但到目前为止，情况并非如此，这是他们面临的唯一风险。

提问者：您能否解释一下为什么这样判断？总体趋势似乎是模型规模越来越大，参数量可能达到1万亿、3万亿、5万亿，下一代新模型甚至可能达到10万亿，但我并不确定。随着模型规模扩大，对内存的需求似乎也会增加。您提到可以通过蒸馏缩小模型规模以供使用，能否进一步说明模型规模和模型蒸馏的发展趋势？

回　答　者：对于Cerebras而言，关键在于每个令牌所传输的字节数。正如您所说，从公开信息来看，模型的参数量确实在增长，但每个令牌所传输的字节数并未以同样的速度增长。这个指标非常重要，不能只看令牌数量。即使模型规模越来越大，每个令牌实际传输的数据量也没有显著增加。因此，我认为这一趋势最终会趋于平稳。

提问者：这是否得益于MOE架构？也就是说，尽管模型规模越来越大，但在一次推理过程中，可能只会使用极少数专家。

回　答　者：是的。MOE可以使容量曲线保持平稳，因为并非所有部分都会同时被激活。这也是我提到蒸馏时所表达的含义。模型规模在扩大，但成本基本保持不变。您的理解是正确的。

提问者：关于Cerebras，在什么情况下，它可能仅仅成为一个……

提问者：在什么情况下，Cerebras 可能只会成为更广泛的异构计算系统中的一个组件，而无法作为独立平台获取显著价值？我们看到 AWS 已披露双方的合作，Cerebras 系统可能会被 AWS Bedrock 采用。您认为 Cerebras 最终会成为 AWS 推理硬件的一个组件，还是成为具有较强竞争力的独立推理硬件平台？

回　答　者：我认为可能会是后一种情况。最终，Cerebras 可能会像 Groq 一样，成为更大生态系统中的一个较小组件。它们的规模过大是一方面，但确实有可能最终只成为更大生态系统中的一个组件。我们此前讨论的趋势可能是其面临的一项风险，并可能促使这种情况发生。

您提到的 AWS 合作为 Cerebras 带来了一定助力，但还需要观察未来是否会有其他超大规模云服务商与其合作。Cerebras 的护城河在于晶圆技术，但仅凭这一点，不足以支持其长期保持独立地位，除非它能够实现业务多元化。按照其目前的业务模式，成为更大生态系统中的一个组件是有可能的。

提问者：Cerebras 还披露了与 AMD 的合作，AMD 也确认了这一消息。目前，Cerebras 已与 AWS 和 AMD 建立合作。您认为未来它是否会与更多 GPU 或 ASIC 厂商合作？

回　答　者：会有更多合作。除了一些大型公司，例如 Nvidia，其他公司都可能从中受益。Intel 甚至也可能受益，S 也有可能。未来开展更多合作是可能的。

回　答　者：我认为，Cerebras要实现进一步增长，也需要尽可能多地建立合作伙伴关系，这正是他们目前努力推进的方向。

提问者：我还有最后1个问题。在AI加速芯片设计和内核开发方面，定制ASIC客户通常能够以多快的速度迭代和演进其解决方案？

回　答　者：您指的是从规格定义到RTL的过程吗？

提问者：是的。您刚才提到，OpenAI从设计到流片大约只用了9个月，这已经明显短于以往的硬件迭代周期。未来这一周期将如何变化？对于AI芯片设计人员和模型开发人员而言，未来的迭代周期大概需要多长时间？

回　答　者：目前，以OpenAI为例，周期已经缩短至9个月，而且这还不包括芯片启动等环节。假设启动还需要5至6个月，那么整个周期约为1年半。我预计，得益于AI，这一周期最终将缩短至5个月以内，整个流程会变得非常快。

我了解到，已经有一些完全由前谷歌员工创立的公司。例如，Recursive Intelligence由Azalia和Anna Goldiv创立，他们曾是来自DeepMind的Jeff Dean的学生。因此，我确信整个周期将缩短至6个月以内。

RTL编码和Verilog编码目前已经在一定程度上实现了自动化。借助智能体AI工具，代码编写速度非常快。过去需要多人参与的验证工作，现在也可以由AI智能体完成。以往在一个芯片项目中，可能需要20人专门从事设计验证；现在不再需要这么多人，而且验证速度也非常快。

我认为，最终瓶颈将来自制造端。台积电、三星或英特尔能否在短时间内为所有客户完成流片，将成为关键问题。相比之下，从规格定义到流片的过程会变得非常快。正如我所说，这一流程主要由软件驱动，而在AI的推动下，软件开发成本目前已经显著降低。因此，周期将从12个月或18个月缩短至4至5个月。

之后的问题在于，台积电能否为你的芯片完成流片。届时会有大量芯片同时进入流片阶段，而台积电无法同时为所有芯片流片。因此，我认为制造端将变得非常重要。这也是英伟达首席执行官Jensen与台积电保持良好合作的原因。他早在3年前就已经开始这样做，因为他知道这一趋势即将到来。

提问者：那么，如果台积电将成为瓶颈……

原文转译

载、转载、转发,引用本次会议内容,否则由此造成的一切后果及法律责任由该机构或个人承担。 本公司保留追究其法律责任的权利。 感谢您的理解和支持。 All right. Um, all right, uh let's start. Uh, first of, first of all, uh expert, uh I'm very uh glad to have you here and uh, first of all, could you please share with us your uh view, your your opinion on uh the GPU and ASIC technology road maps and architectures? uh, what are the advantages and disadvantages of different GPU and ASIC uh ASIC ASICs? Yeah, so, um, like uh GPUs uh in general are like for uh uh like a breadth of workload, not like for speci specific workload as you know. Uh, while ASIC is uh uh like more like uh custom and less flexibility. But you do get like higher performance for the same cost. Uh but it's like mainly for uh uh target uh workloads, right? Like a specific workloads. So you are like having a trade off basically between uh um what like the breadth basically of what all you can do with it versus the uh the execution uh is performance for the same amount uh or same power. Okay. Um, um, do you think that uh uh are there will be uh are there will be a trend that um different ASICs will do will uh be uh will do different like will do different tasks or will be more suitable for different workloads? Like, uh, there were uh TPUs, right? There were Google's TPU will be uh there were like uh I think an inference TPU and and training TPU and also for uh for Nvidia, Nvidia acquired uh Groq. So Groq has its uh LPU for maybe for for decode. And also there there is Cerebras. Uh their their wafer uh is maybe uh best suitable for uh fast inference for for decode. So do you think that that uh this is a trend that different ASICs will be uh more suitable for uh different kind of different kinds of workloads? That's exactly right. Yeah, so this is the trend we are going in. It's like um it's called like heterogeneous inference, right? So, as you said, right, Nvidia acquired Groq, Cerebras uh are working with I think AWS. So, the reason is that um like if you think about it, the uh Nvidia, for example, right? They already have GPUs, which can do uh some tasks very aggressively, right? Like when the user asks questions for example, the first thing it does is it collects all the data at once, right? Into memory. So GPUs can do that very fast. But there are things which GPUs cannot do as good as uh what uh like uh uh like companies like Groq used to do like the like the LPU, for example, for decode, right? So they can like uh combine those two together. So which uh for example, they did already with the merger of uh like with the acquisition or acu-hire of Groq, right? Where they took the co-founder and president. So, um and uh similarly like AWS also did and I feel like uh more and more companies might uh do that going forward. Uh like because like there is a trade-off as I said, right? So, uh for I think uh OpenAI may be next, I'm not sure. Like you said, right? Jalapeno, I think. So, right now, it's like only inference, but we can expect uh this to continue uh going forward. Like at at least that's what people think in Bay Area right now. Okay, got it. Got it. Okay. And then, um, how do we compare different GPU or or ASICs, like are there any any common benchmarks for or metrics or specifications for that can be used to compare different GPU and ASICs? How do we under I mean maybe could you help us understand how to compare the different ASICs? Um, so, yeah, I think uh it's like uh for the same user uh experience, I would say, uh how much is the like uh like how many tokens per second you can have? Uh or like based on power. So let's say tokens per second per power, like whatever that unit is like what, let's say. And uh also the throughput, um like uh basically the total cost of ownership of having that platform run uh per uh per what basically. Uh so TCO is like as important and I think it's the 50% cost of operation actually. Like you can make a a system with a um like very high performance, but performance per what or the TCO is the major matrix. So, that is one uh criteria that is being used right now. So it's called like tokens per second per what. So that is uh driving basically which one is better or not. Like there are other criterias as well, but that's one of the main criteria. And it's also based on like user experience. What that means is like a user can be asking like um like some users might be aggressive, right? They might be asking very fast, for example, versus there are some users which are like like common people, right? They they they don't care about uh like they're not using it to create applications, for example, right? So when it is compared, it should be compared for the same type of user experience. So I think that's one of the uh like basic benchmark that people are using right now. Is is uh yeah, I think uh and also how fast you can deploy basically. Like uh uh for example, like Cerebras, right? They they they do have like a large wafer, but can they um deploy new models on it, right, quickly? So that is another criteria or benchmark. The time to deployment. Got it. Got it. Understood. And um yeah, and and another question would be that um um so so I think there were there there there were different uh different scenarios or different applications, different kinds of workloads. And typically, uh what kinds of workloads will require uh like different kinds of uh different kinds of uh uh features or or or metrics? Like in in what scenario uh speed like the the tokens per second per user matters matters most and in in what kind of uh scenarios uh uh from which will matters most and also in what kind of uh in what kind of scenarios, um as you said, um um time to time to deploy and time to deployment deployment matters most. So, could you share with us a little a little bit on that? Uh yeah, I can. So, let's say um uh we are having like a chat bot or something, right? like chat GPT, right? So, then uh what matters is like uh time to token, right? So, then the latency is important. Um like time to latency for user per second, right? So, that's why the Grock or Cerebras territory is important, right? because of that. Then um uh another criteria you said, I think throughput. So, if you have like documents that you need to process, right? So, then uh like latency might be not that important, but uh for uh dollar per tokens might be important because you're processing lots of data uh without depending on the latency which user uses, right? The other thing is like uh time to deploy. So, this becomes important when some like a startup or company has to use some new like they have to go with new company, right? with Cerebras or Grock that time, it becomes important if they can run their uh models on their new on this new platform, right? So that's like enterprise level. Then yeah, and in some applications like video, like if you're asking LLMs to generate videos and all then uh what matters is like peak tokens per second because there is like uh just like power has that peak graph, right? In dynamic power. So here also you have that uh like if you plot that tokens uh with respect to time, you will see that there is like a curve. So it peaks uh when when you're running videos, it peaks, right? When you're generating videos using LLM. So that that is very important for like real-time video or like maybe voice applications. And um yeah, I think uh other thing, yeah, and when you are training or then then uh mainly like flops uh matrix matters basically. Yeah. Okay, I got it. Okay. Uh how about how about AI coding? Um and and also another question is that um we we we see that um sometimes the same customer a same company like like cognition will uh purchase cloud service from like from Nevius and also from Cvas. Uh so so what do you think is is this the difference? Uh because I think maybe cognition is like a is a is a maybe a player in in in AI coding or some something similar to that. And why uh cognition will purchase like uh cloud services from from from different service providers. Uh I see. So like when you say AI coding, like you want to know which matrix is important basically that time, right? Yeah, yeah, yeah, that also yes, that would also help. Yeah. Yeah. So it's like um uh like it will be mostly latency only. So, but it's like uh basically we because you need response to be fast like when someone is coding, so I would say it's in like uh interactive latency. Um and your context uh needs uh like large memory. because people don't know what they want to code, they go back and forth, the memory, the context, like how many tokens can it have, like whether 200k or 1 million becomes important. Uh so that is uh one thing and also the reasoning, uh like how like that is another benchmark, right? The the quality of reasoning itself. So that is also important for like when the user is coding. Does it make sense? Okay. Uh, please help me uh understand uh uh you said that uh people maybe people don't know what they want to code. how do I understand that? So, yeah, so let's say, um um like let's say chip designers, right? So they ask LLM first, uh hey, can you write me this piece of code, right? Then uh LLM comes back that it says that do you want to do different way or like uh um so then the user goes back and forth, right? It gets some code from AI. They try that code on their system, like let's say chip designers, right? They use innovas or like writing very log system, uh very log. So they use that and then they find error and then they go back and forth. They copy, paste, check the output what the AI model generated. And so that's why the context usually becomes large. because of the copy pasting that goes on and on like back and forth from what the AI is giving versus what the tools are consuming. So that that's what I mean by uh why the it needs big memory when like people are coding because lots of copy pasting happens between users and to the AI chatbot. Uh, okay. Okay, got it. Okay, so and also, uh in this case, latency latency also matters, uh am I am I correct? It does, yeah. Yeah, it needs a low low latency. Yes. I think that's important for this case, yes. Okay. Okay, got it. Okay, and also, yeah, and um uh regarding on regarding uh fast inference, I think there are several players like like uh faster inference uh Asics. There arebus, there are uh, and also some maybe some other smaller startups. Uh what what do you so so first first of all, what uh how much is the uh uh what is the market size? What is the market or what is the market potential of fast of fast inference? I mean, uh currently and also maybe in the future. How how much percentage of the total inference market will be uh fast inference? Um so that is actually difficult to say, uh but like based on growth, it can be a pretty significant market, right? Like maybe 50 billion, 50 billion dollar, like in let's say five years. But uh it also depends on what the overall market size is, right? But uh yeah, so it it is it is very like um it's it's growing very fast is what I can say. Uh especially because of uh like AI coding and then uh real time agents, right? That people are deploying. So, like just like what Meta did with muse. So so I think I think what Meta did with muse is proving that people do need like there is a huge demand for real-time agent, right? So Meta just launched this muse agent and the downloads exceeded what OpenAI was doing within like one week. Yeah. Okay. Okay. Um, okay, let me uh ask this question in an in in another way. So like maybe uh for like for for for Cerebras, for for for Groq, they claim that they can uh they are tokens per per second per user could be uh up to like over 2,000, 2,000 tokens per per second per per user. And maybe I'm not sure, maybe in in uh general AI coding scenarios, uh maybe I'm I don't know, maybe you you you can tell me that. I I maybe like uh 300, 500 tokens per second per user will be good enough. I'm not sure. So so how do you so, um, could you like estimate what are the what what is the percentage of the like uh, ultra high, ultra fast inference uh, in in the general inference market, like what what is the percentage of of the of the of the of the needs of of the needs for like uh uh token speed over like over 1,000 tokens per per second per user. Yeah, so I mean, yeah, do you understand my my uh uh my my question? So, yeah, you are saying that uh like even though Groq and Cerebras are claiming that it can do 2,000 tokens per second, but Yeah, yeah. how much does usually, how much do usually people need, right? Yes, yes, yes. Yeah, so I think um uh this uh inference like even though they are claiming uh 2k, it's more like right now slightly less, right? In general it is like 400, 500 only. But uh it's not I feel it's not a requirement. It's much more than what is needed. Uh like that's why I think cheaper uh um like because because users like don't realize the difference, right? at at a certain point. So that's why like even though like um it's it's mainly the agents, right? which are running the loops. That's where it becomes important. But for uh I would say at enterprise level, it's more important. So for 2K uh token per second is very important when it's like at enterprise production level. where company's employees are using it to code their running agentic AI solution or loop like jetski from Google overnight, right? They let it run. It is and they when they come back in morning, they want their tasks to be done. So that's where it becomes important, but uh for bulk of like inference market, I think is throughput throughput is more important than speed, I feel. Okay. Yeah, unless uh it's like a live agent or something, right? Like a voice agent where you need it to be fast, right? But um yeah, I I that's what I think. Yeah, is more like throughput matters more than speed right now. Okay. Okay, got it. Yeah, and yeah, so maybe a basic question from me is that uh what is the key difference between throughput and tokens per second per user? So throughput is like uh so you see right um when you uh like the LLM is doing some calculation, right? So then it's like that like total tokens it is producing. thing per unit time, right? Uh, like, because it, uh, so there are two ways, right? So user types response, it goes back, right? And then it is doing some, uh, calculation, right, at the back end. So, so, so that's the difference. So, to to, like, tokens per second, when I say, it's mainly the, uh, how fast it is going to to where the calculation is happening, right? And then throughput is like, uh, basically overall, like, uh, how much, uh, tokens per users are being produced, like, or we can say like the capacity. Like, uh, yeah. So, so one thing is that you you will notice, right? The other thing you will not notice basically. Like, because parallel requests can go, right? Like batching. So, there is like that's that trade-off basically. Okay, I got it. Okay. Okay. And, um, um, uh, how do you see the like the the the future, future potential or, uh, outlook, growth outlook for uh, Cerebras? What do you think it would be like a a much larger company in the company in the future? Would be would be would it would it be like more much more, uh, um, um, commercially successful in the future? What would you think? They have some competition from the this, uh, Nvidia Groq thing. But, uh, overall, like, in the investor community, in general, the they are saying that they will do good. And they, the solution they have is not easy to replicate as well. Because I, I, I'm pretty sure you know, right, what they did, like how they created that, right? So, they just took the wafer and did not cut into die, right? So the, so it's like a, consider it like a chiplet, right? But it's the most innovative thing that happened, I think, in terms of like, uh, physical design of the chip, where they just use that entire wafer, and the chips are already, the dies are already connected to each other, right, on wafer. So, now that's why it's difficult for any other company to easily do that. So, that's why they have like a long moat or defensibility for at least, uh, like at least three, four years, I would say. Uh, they they have to, of course, like, uh, get more customers and prove that the model is efficient, but so far based on like what Wall Street is saying and what people are talking in Bay Area, they, the outlook is really good. They, I think have some deal also with the, um, like OpenAI. So, but but we have to also think that OpenAI itself is now creating Jalapeno, right? So, will they work with them or not? Um, like, will they do what, uh, um, like, will they do something like what Nvidia did with Groq that that's still to be seen. But there is some, like, I would say bear case because of that. Got it. Yeah, so, um, yeah, so what what is the, uh, like what what is, yeah, because they have because Cerebras has a very big deal with OpenAI and OpenAI just released their, uh, jalapeno. Um, what would be the relationship between these two kinds of ASICs? Uh, Cerebras like WSE and and OpenAI's jalapeno, will they like compete compete with each other in the future or I'm not sure I don't know, maybe or uh will there be any room for like cooperation or collaboration between the two kinds of ASICs in the future. I think um, it depends like uh short-term and long-term. so like they do need um, like OpenAI is still heavily dependent on inference uh right? So, that's what they they are getting from Cerebras. So, I think that's why in like short term, I don't see problem. But in long-term, there will be competition because of uh um like they they will still work with Cerebras maybe. but uh yeah, I I I I think uh it it the time will tell. I would say. There is uh problem like there is a potential for uh Cerebras to take hit at some point. Uh, the the the the other thing is like OpenAI can be very um innovative and they can create chips much faster, right? So, they they they will like but I I think the the wafer solution that Cerebras has that that itself has some mote in it, right? So that's why it is still to be seen because OpenAI might not be able to do what Cerebras did at least on the wafer side by themselves. So Okay. Yeah, yeah. Okay, got it. Okay. And then um, okay, um and then from the from the perspective of uh software ecosystem, how does uh Cerebras compare with uh with Cuda? and also maybe yeah, also maybe uh Google, Google's um TPU and AWS. uh how how do these other players compare with uh Cuda? Um so Cuda like is the like I think I would say is best uh in general because like is like more than I think 10, 20 years, right? I think 15 years maybe. So they have like uh a lot of uh data, like a lot of customers have used it already and they have gone through lots of iteration. So, uh I cannot speak about like uh TPU because of the agreement but um I can talk about maybe AWS. So, so AWS is uh better than Cerebras. Um because uh like the in terms of model coverage, uh AWS ranks higher than Cerebras. Cerebras is like good for um very simple uh models or or I would say only like it's easy for whatever they support it's easy to do it, right? But the coverage is very low. and not many people are uh familiar with that software stack. So, so I would say Cuda is number one then AWS is solution for software, then it's Cerebras. Uh But uh like Cuda is like kind of plug and play, right? It is very fast to just run whatever people want on it. But um, yeah, I think that's what that's why AMD could not for example catch with Nvidia till date, right? It was not never about their chips, it was always about the software solution that Nvidia was providing, right? Even though the AMD's chip was even better than what Nvidia chip was. in terms of like power and everything. And even the even the layout of the chip. But they still couldn't catch up still, that's because of CUDA. And I think that that mote is going to stay with Nvidia. They have like a huge talent in that department, they spend a lot of money. And uh, so that is the reason some, like some companies what acquired, right? I think AMD acquired someone as well. so that they can create that software stack. So, Cerebras has I think a lot of work to do in that department. And but uh, yeah, I cannot talk about TPU, but yeah, so yeah, that that's my answer. Okay, got it. Got it. Okay, um, so, um, what what's the difference, uh, I mean, what's the difference between CUDA and other players from the perspective of end users? Like, I mean, maybe from the perspective of of developers, uh, CUDA is much more, like much, the ecosystem is much more mature, sophisticated, and they can like, uh, maybe they have as you as you said, they have already iterated uh for many years. And and then, um, could you help me understand from from the perspective of of end users, what's the difference between like between the between CUDA and other other software uh stacks, like, uh, is the difference uh, like like for example, maybe I'm not sure, I'm please help me understand. Maybe uh, some a difference, a difference will be like, uh, CUDA can uh support a new model at uh on day, on day zero, and other players will will be able to support a new model, maybe like in in in a week, or maybe in a month. Yeah, what what what's the, what's the difference from uh, from the perspective of any users? Yeah, so it's like um, uh two days, two, three days for let's say CUDA, right? or even less, maybe within a day as well. For uh Neuron, Cerebras, it's uh, it can be days to weeks. Uh from the user perspective, right? Then they have to debug also, something not working. Uh, and then they go to the IT team. So there is a lots of back and forth, there are loops, that takes time. So, I think that's why users uh prefer CUDA. Then if some error happens, uh then CUDA might have a solution for that because someone saw that uh before, but uh maybe we are seeing first time, let's say if we are running Cerebras, that that issue might have happened first time with some user, right? So, they are stuck with that. Then, um, uh, uh it it is also like uh from company perspective also there are criterias, right? Now, if you have to hire, let's say Nvidia, they they can hire an engineer who knows CUDA, but they cannot hire someone who knows Amazon or Cerebras solution. So, so that that is another issue, right? Then, uh, yeah, I think uh, also many problems have been solved before on CUDA. So, you can, you might have like some recipe already available for some some problem people might be solving, right? So I think all these uh combined, right? I think does matter. So, is that's why I think, uh, I mean, yeah, that that's my answer. Okay. Okay, got it. Um, And uh also another another question is that um are all, I mean, are all the, uh, from tier models trained trained like in the Cuda ecosystem or like in under I mean, in in the in the Cuda uh, software stack? is the is my understanding correct? Um, yeah, I think, um, you are right. It is true that I think maybe I would say 90% models are trained in Cuda. Uh, like I think Open AI did on that, uh, Anthropic, uh, maybe not. On Anthropic, I don't think it was trained on Cuda. And, uh, Open AI is, uh, I think they are going with AMD as well now. So, uh, it used to be, I think more than 90, it was 95% at some point, but then now it is, I think 90 or less. Uh, because of this growth in inference. So, yeah, I think Cuda was go-to place. It it is still is majority. But, uh, other companies are starting to catch up. Because of the same metrics, right? Performance per watt. So, other companies are catching up because people are realizing it that they really don't need like at least enterprise is realizing that they really don't need one, uh, like common solution for all models, right? When, like, I don't know if you heard about the company called MatX. So, they they just raised, uh, I think they are 10 billion dollar worth now. So, that's what they are doing. So, they are creating, they are claiming to have the best inference chip. Right? Let's see if that works out, but Okay, you said what's the company? Mat, MatX. MatX. MatX, yeah. Okay. Yeah, so they you should, uh, yeah, sir research about MatX. So they will, I think become as big as maybe what Groq is or even bigger. So they are very they are actually X Google people. He he was the founder of TPU, right? The main architect of TPU, Google TPU. He left and started this company. He took like lots of other Google employees with him. And, uh, they are doing lots of like custom work and they are targeting this, uh, uh, throughput basically to make the the best inference chip. So. But yeah, anyway, coming back to your question, yeah. So, Cuda it is right now for 90%. Okay, okay, got it. Yeah, oh, okay, yeah, yeah, maybe another follow-up on follow-up question on on CUDA. So, yeah, so so today, uh, so in I mean, in in training, uh, Cuda is is still, uh, dominant, right? Maybe maybe 90% uh today. Yes. Okay. Uh, Okay, and and um so typically, I mean, for other players to uh support a like a new a new frontier model, uh, which which is trained maybe in in the Cuda ecosystem. How much time does it take to support the new model? Like for maybe for AMD, uh, how many days, uh and and maybe how many manpower, how many engineers are needed to support the new model? And also maybe for for for Google, for for AWS, uh, for for other, uh, for other, uh, players, for other ASICs. Uh, how many, how many, how much time and how many uh engineers will be needed? Yeah, so, like you were asking uh, uh like on CUDA, how much uh engineers like uh will it take and time? And on AMD, how many, how much days and time will it take, right? to train? Yes, yes. Yeah. So, I think these labs uh like do iterate weekly or two weeks, right? So, we are like on AMD and all others, it can take like two months, uh basically. But uh CUDA might be like much faster. So, Oh, let me confirm. I'm sorry, let me confirm. I I mean, the time to um, how much time does it take to deploy? Uh, I'm not asking training. I'm asking uh how much time does it take to deploy the new model? I mean, maybe for CUDA, maybe for like any uh uh, I mean, in CUDA, in the CUDA ecosystem, maybe it just take one, within one day, the new model could be uh deployed. But if we deploy that new model on like AWS Trainium or or the Google TPU or or Cerebras uh uh uh Cerebras systems, how much, how much time and how many engineers will be needed to deploy that new model? I see, I understand your question now. So yeah, it might take um like four, five weeks, uh. uh mainly with the kernels, then uh to get the approvals for model to be supported, right? Then, uh, uh like also, yeah, I think uh mainly fine tuning, uh. So, I would say like roughly four weeks minimum. And then uh in in terms of engineers, uh maybe uh five, five engineers maybe max, like the people who have knowledge. Uh, maybe two senior people or three senior people to junior people. Uh Yeah, mainly some some kernel guy, compiler guy is needed. So, Okay. Yeah, I think, yeah, we can say five, yeah. Okay, and uh regarding the yeah, so how much time? I mean, uh we we we will it be the time will the time be different among like AMD, uh AMD, AWS, uh Google, uh Cerebras, like maybe AMD will take one, it will take one week for AMD and it will take like four weeks for for for Cerebras. I'm not sure. So, yeah, so will the time be different? Um, it will like uh roughly be same. I would say just uh Cerebras uh like or So, AMD, let's say will be the I AMD and Cerebras are kind of similar, let's say. Because they they are very close to like uh like CUDA, right? And uh Cerebras has to support the model, but if it does support, it can take same time like maybe two to four weeks. or maybe four to six week, I would say. like minimum four, the reason I'm saying is is like uh you have to go through like uh you're just bringing a new model, right? So you have to go through that whatever the process is, and then AWS is uh longer is what I heard, but yeah, Google TPU I cannot comment. comment on. So AMD is fastest maybe. Cerebras after that and then AWS is a trainium is the slowest one. uh, yeah. Okay. Okay, okay, okay, okay, got it. Okay. Um, Okay, so maybe uh so next question will be um, what is the potential impact of Nvidia's uh, acquihire of Grock? Like, uh, what's the what's its implications to the competitive landscape in the GPU and ASIC market. Um, I think uh it's a very good acquisition. It will eat lots of companies, like, I mean, it will affect lots of companies because of the latency, like the the the one problem which Nvidia had, right, they just uh solved it. They never had problem with the GPU, they had their best, uh, like the pre-fill GPU solution. So, now they we we have like a killer product, right, where you have like the best, uh, pre-fill and you're using one of the best for decoding, right? So, they have the pricing power as well. Someone uh, new who is doing might like if they want, if someone else wants to do that, right, they will have to uh, they cannot compete with Nvidia on pricing as well and also the time to deploy is also fast, right? Um, but this also means that uh the this thing is real, basically is what it means, right? So, we are going where we the people are mixing, people will mix GPU and ASIC because they get motivation that Nvidia did. So that's what AWS, uh, I think is trying with Cerebrus. Uh, so, it is overall helping in that sense because there are not many companies who are doing that, right? But it can hurt. Yeah, I think as long as the overall AI market is growing, it is helping everyone. If the market was constant, not growing, then it would hurt the smaller players. Is what my answer is. Okay, yes. Okay. I got it. Okay. And um, yeah, and also uh uh regarding uh OpenAI's Jalapeno, Um, uh yeah, so what what's its impact on on on on on to current uh competitive landscape. And uh yeah, yes. And that's the question. I think uh it's actually, so I I know the people like who created this chip, right? They're very smart people. They all some of them came from Google. They they they they used their own like AI to create the chip in half the time, right? So, that means they can, if they let's say find some problem in the chip, they can again create it. They I mean, the OpenAI is a company created the chip for the first time and they did it in half the time of what other companies are doing, right? The cycle time, the the the time to create a chip from spec to tape out is 18 months. Uh, and this, uh, OpenAI did this in I think 9 months, right? So, just half the time. And uh, they have, uh, I think they compared with the Blackwell and what they found out was that they are offering 50% better performance per watt compared to what Nvidia is doing. And what or power uh correlates with the cost, right? Uh, when we check TCO, which is like total cost of operation, the 50% of the cost comes from power. So if if someone is saying that the it's 50% more perf per watt, that means their like cost of operation is also 50% more basically. So, that's why I think uh, it's a very big signal that all these uh frontier model companies will eventually make their own Silicon. Because they can do it better than what hardware companies do, because they now have this agentic AI solutions within the company, right, which can accelerate the chip design cycle. Just like what what recursive intelligence is doing. I don't know if you heard about them. They claim to do the entire floor plan and placement in of chip within like a, um, a day. that used to take one week. So, yeah, I think um, they they will give tough competition to NVIDIA is what I feel. And maybe even to other companies like, um, Amazon. Yeah. Okay. Okay, then, um, how about Anthropic? Uh, is Anthropic also like developing their own ASIC at the moment? Yes, they are. They actually just hired uh Amir Salek as their hardware guy. Uh he's also ex-Google. He was the one who created the TPU program, Google TPU, like not he was not the creator, but he was one of the main guy. So they hired him just now and he's now creating a team. But uh, they they yeah. So they're right now renting it, but they have not disclosed it, but they they are going to do it because they hired someone who is going, who has done that before. So, I think 100% they're going to build it. Okay. Okay, got it. Okay, then um, yeah, we uh, we see uh I mean we have seen that uh we we we got this information that um, Anthropic from the uh, earnings call of Broadcom. Uh we knew that Anthropic has signed a I mean a a very large deal with Broadcom. And Broadcom Hawk 10 of at Broadcom said that Anthropic has like uh 5 gigawatts of demand uh for like for for signed with Broadcom. So is what what do you think is is that 5 gigawatts um Anthropic's ASIC or like or or or Google's TPU? because we know that Anthropic may uh uh Anthropic use Google's TPU. Yeah, so, so, so what do you think? Did the Anthropic's 5 gigawatts of demand uh uh uh um uh, uh is like like is it is it is it Anthropic's ASIC or is it Google's TPU? So that's uh I I cannot comment on the Google TPU because I'm part of the TPU team, but I can say that this large number is not it is a data center scale number. It's a very big number. So, so that's what I can say. Okay. Okay. Alright. Um, okay. Uh so um, all right, let's move to the next ques- question. So, yeah, so what what are the key like uh key sources, key competitive advantages of cerebrus, like uh patterns, uh IP, knowhow, um um maybe com compiler technology, uh customer relationship, etc, etc. So what what what do you think are the key advantages of Cerebras? Um, I think the key advantage is the the the the the mote that they came up with, right? The way they developed it, the the wafer level thing that they did. I'm I think you might you must be familiar with what like a wafer is, right? When they create these chips, they um usually the TSMC or whatever, right? They use the like one big wafer and then they cut it into pieces, that becomes die, right? And then people use those dies to either build chiplet or they assemble it on the package or board, right? But what that does is it reduces the latency because within a single die, you can have a very fast latency, right? Because it's just a single die, everything connected through metal layers. But now, when people use chiplet, they have these chips which are separated, right? Uh, and then when these chiplets talk to each other, they use like um interconnect, right? Which might be like copper or these days they're using light. But the advantage Cerebras has is they innovated this new method, right? of manufacturing where they put everything on a single wafer and they are using that wafer itself, right? So that is their main moat, I would say. The other thing now is, uh because of this, they are able to uh like map lots of models, like or the or the the size of the models as well. So, excuse me. So that's why they can use uh like that's where the advantage is for them. And then they can deploy this at a very large scale because of the wafer thing, right? So, uh, and and the deal that they did with companies prove that this is not like just some college technology, right? Or college project. It's actually industrial level thing they did. It is scalable. It can be produced, right? And um in terms of patents, uh I no, that's not their main advantage, I would say. Uh, it's just uh first to market is their advantage. They did it before anyone else did it. That's where their advantage is. Okay. Okay. and um um Um so and also so next next question would be um um what type of what types of of changes in AI model architectures could possibly could positively or negatively impact cerebra's wafer scale SRAM based system. Uh, so there, yeah, I think that's a good question. So, um like if if you have like big model, like people are going with the large context right now, right? And there is a race towards getting. So that makes cerebra shine. Uh, but now it's possible that uh in future it's uh like memory is not that important, let's say, right? where people are um like doing some distillation, for example, right? So, I think that's when they might uh yeah, so like when the industry goes in the direction of less memory, so then their their important.. will go down basically. But uh so far that's not the case, but that's the only risk to them. Uh, okay. um, Okay, so uh could you could you please help me understand uh why you said that I mean, so, uh so I think a general trend would be would be the uh model will become like larger and larger. I mean, the parameters will be like one trillion, three trillion, five trillion or maybe I don't know, maybe what's the next what's what's the size of the next generation of uh new model like maybe 10 trillion? I'm not sure. I don't know. So yeah, so so maybe the the model will be larger and larger and uh it seems that the demand for memory will be also like like larger. So, yeah, how and you said that maybe this distillation, right? distilling these models and making these models maybe smaller to use. Yeah, so so could you please um um, elaborate a little bit on that like uh what's the like what's the trend of the the the model model size and also uh this distillation? Yeah, so uh like I think so what matters for Cerebrus is uh let's say if we take example, so you're right, the if you see the headline, right, that's what it is that the parameter count is growing, right? But uh the bytes that uh bytes streamed per token matters for them, right? And what we are seeing is that the trend is that bytes streamed per token count is not rising as fast basically. So that matrix is important, right? Not just the token count. So, Okay. uh does it make sense? Like even though models are getting larger, right? But uh uh like how much actual uh data is streamed, right? per per token, let's say, that that number is not rising as significantly. So that's what I was talking about when I say that it will flatten, right, at some point. Oh, okay. Oh, um, Yeah, uh, okay, I think I, yeah, I I guess I'm I I I got it. I got I got some of your point, I think. Maybe is it like is it because like uh like the the the the MOE architecture uh enables that. I mean, the the even though the model becomes larger and larger and but maybe in in like in a uh uh inference session, there will be like maybe a very very few experts uh that will be like used. Yes, that's right. Basically, yeah, you can say that MOE make sure that basically the capacity, right? Uh, yeah, I think it is keeping, yeah, I think that's what is happening. It's keeping that curve flat basically. like since not everything is getting activated, right? Uh at once. So that's what I meant when I said right, uh distillation. So, basically the cost is uh let's say same, but your models are getting larger. So, yeah, I think you're right, yeah. You you got the point. Um, okay. Okay. Yeah, and also another question on regarding Cerebras. So, um, on the what on the what circumstances could Cerebras become merely a a component with a broader heterogeneous computing system, rather than capturing significant value as a standalone platform. Like like, yeah, we we have seen that AWS AWS and has uh disclosed the their collaboration. And systems will be adopted in the AWS uh bedrock, I think. Yeah, so so so so how what do you think? That that's the question. Uh so can you please repeat? So you what do you mean like in what circumstance re would? Yeah, so uh yeah, so could uh I mean could so under what circumstances could become become just a component within the broader computing system? Or uh rather than rather than uh as a standalone platform capturing uh much more significant value. Like like like what do you think like will become a just a like just a component for example, a component of uh AWS uh inference inference hardware. Or cerebrus will be uh like a very competitive standalone standalone inference hardware. Uh I think it will be the latter case. So eventually uh cerebraus might become a smaller component in just like what happened with like Groc, right? So I mean they are too big. That's one thing, but it is possible actually that they like they just become a component in a bigger ecosystem. and uh uh I think this what trend we discussed, right? might be one risk to them, which might cause that to happen basically. So there I think uh the the the AWS uh thing that you mentioned, right? gave them some uplift, right? But we need to see if uh in future we see some other hyperscaler going with them. But I think there is a risk, like their mode is in, as I said in their wafer technology, but that's not enough for them to stay stand alone for very long term unless they uh like diversify their business. With their current business, um it is actually possible. Yeah. Understood, understood, yeah. And they also um disclosed that um they they they are they will collaborate collaborate with AMD, I think. AMD also confirmed that. Yeah, so how about the uh what do you think? What do you think like like we'll uh like in the future collaborate with more more GPU or AC players. Uh currently they have like collaborations with AWS and and AMD. Yeah, what do you think in the future? Will there be more like more more collaborations? They uh they will, yeah. Like um so whoever is uh like except let's say big um like maybe Nvidia, other companies might benefit from that. So, like maybe even Intel can benefit uh or like maybe S also. So yeah, I think it's possible. uh the I think Celebrus uh needs that as well. um to grow from here and I think that's what they're working on to make as many partnerships as possible. Okay. Okay, got it. Okay. Okay, one one last question uh from me. Uh so, uh in the context of the AI accelerator chip design and kernel development, how quickly can custom AC customers typically iterate and evolve their solutions? Uh, so, you mean uh from spec to RTL? Yes, like Yeah, like like you you said that um you mentioned that OpenAI just spent like nine months to yeah, to from from maybe from a design to to tape out. And that's yeah yeah, so is already significantly shorter than previous hardware like hardware iteration cycle. so what's the what's the future will be like? Like what what how much time, how uh how much time does it take the A the A uh designers and also the model I mean I mean what's the like the the cycle the cycle what's the iteration cycle in in the future? So, true. So, currently, like, let's say we are already down to 9 months, right, with open AI. And not including let's say the bring up and all. So let's assume bring up is another like five to six months, right? So let's say one and a half year. I I am predicting that uh it will go down to like eventually less than 5 months. That's what will happen, like because of AI. It will become like super fast and there are companies that I know already which are founded by former Google employees only, right? There is a company called recursive intelligence. It was founded by Azalia and Anna Goldiv, who were the students of Jeff Dean, right, from Deep Mind. And so I I I know for sure that the the cycle time will come down to under six months. because uh like RTL coding, right? And very log coding is already kind of automated, right? Uh the tools they have, the agentic AI tools, they can write code very fast, right? And the verification which used to involve like multiple people is being done using AI agents now. Uh like people like it used to be like 20 people, right, uh in a chip program, doing just design verification. Now you don't need that many. And it is super fast. Also, I think the bottle neck will come from the manufacturing side eventually. If TSMC or or Samsung or Intel, right, can they tape out for everyone uh in a short time. I think that's what will become a bottle neck rather than from spec to tape out. I think spec to tape out will become super fast. As I said, because it's all software driven and software is cheap right now because of AI. So you go from like 12 months cycle or 18 months cycle down to four, five months, right? And then plus can TSMC tape out your chip? Because there will be so many chips taping out all at once. And TSMC cannot just tape out all chips at once, right? So that will become important, I think, on the manufacturing side. And that's the reason Nvidia CEO, right, Jensen collaborated very well with TSMC because he knew like and he did that three years ago itself because he knew it's coming. So, yeah. Okay. Okay, got it. Then, if TSMC will be the bottleneck, then