Firehose

I have a lot of mixed feelings about AI and LLM technology. I’m fascinated by its effect on our profession, excited by the potential gains in productivity - and thus the products we could rapidly build. On the other hand, I’m fearful of the…

The article discusses a recent escalation in AI safety concerns, following Jacob Coxon's resignation and the resulting preference cascade. This has led to increased scrutiny of AI companies, with Anthropic CEO Dario Amodei and OpenAI pledging to take steps towards safety. As a result, people's estimates of AI's potential risk to humanity have roughly doubled, from ~15% to ~30%. AI summary

Ben Thompson interviewed Joanna Stern about the iPhone Duo and AI for normal people, discussing the implications of Apple's AI-driven products on the market and consumer behavior. Stern highlighted the potential limitations of Apple's AI approach, citing concerns about data security and the risk of AI-powered products becoming too complex for normal users. AI summary

AIUC's CEO, Rune Kvist, has raised $40M in Series A funding, backed by a list of prominent industry advisors, to build confidence infrastructure for frontier AI through standards and insurance, addressing the growing concern of liability and risk in autonomous systems. The company's mission is to make AI deployable and trustworthy by providing a standard for agent security, safety, and reliability, and by insuring AI systems against potential risks. This approach aims to overcome the current bottleneck in AI adoption, which is driven by trust and liability concerns rather than capability. AI summary

Tool: Gemini Live audio Google released Gemini 3.8 Live and 3.8 Live Extended Thinking today - two new speech-to-speech models that are a similar shape to OpenAI's GPT-Live family. I pointed GPT-6 Astra Extra H

Anthropic has disrupted numerous attempts to misuse Claude, a large language model, by malicious actors, including attempts at biological misuse, conventional weapons development, and illicit distillation. Notably, Chinese labs have been found to have systematically attempted to distill Claude, with some using thousands of new accounts created with stolen credit cards and API keys to harvest user data, raising concerns about the misuse of user data. AI summary

Here is a summary of the article in 3 plain sentences for a developer/AI-ML audience: Open-source AI models have gained significant attention in recent years, with leading models coming from Chinese labs, and their adoption is expected to continue growing, with the open-closed model gap narrowing to around 4-6 months. However, concerns around safety and cybersecurity risks associated with open models, such as distillation, remain, with some arguing that distillation is a key factor behind Chinese models' performance, while others argue it is overstated. The US government is taking notice of the trend, with regulatory attention focused on companies using Chinese models, and experts advocating for a national AI cybersecurity policy to address emerging threats. AI summary