Posts

Showing posts from January, 2025

Chain of thought ✨

"Recent studies have discovered that Chain-of-Thought prompting (CoT) can dramatically improve the performance of Large Language Models (LLMs), particularly when dealing with complex tasks involving mathematics or reasoning.  "Despite the enormous empirical success, the underlying mechanisms behind CoT and how it unlocks the potential of LLMs remain elusive.  "In this paper, we take a first step towards theoretically answering these questions. Specifically, we examine the expressivity of LLMs with CoT in solving fundamental mathematical and decision-making problems .  "By using circuit complexity theory, we first give impossibility results showing that bounded-depth Transformers are unable to directly produce correct answers for basic arithmetic/equation tasks unless the model size grows super-polynomially with respect to the input length.  "In contrast, we then prove by construction that autoregressive Transformers of constant size suffice to solve both tasks...

Limitations of multi-layer transformer

"In this work, we prove the first unconditional lower bound against multi-layer decoder-only transformers .  "For any constant L, we prove that any L-layer decoder-only transformer needs a polynomial model dimension (nΩ(1)) to perform sequential composition of L functions over an input of n tokens. As a consequence, our results give:  The first depth-width trade-off for multi-layer transformers, exhibiting that the L-step composition task is exponentially harder for L-layer models compared to (L+1)-layer ones;  An unconditional separation between encoder and decoder, exhibiting a hard task for decoders that can be solved by an exponentially shallower and smaller encoder;  A provable advantage of chain-of-thought, exhibiting a task that becomes exponentially easier with chain-of-thought. "On the technical side, we propose the multi-party autoregressive communication model that captures the computation of a decoder-only Transformer.  "We also introduce a new proof...

Tülu 3 405B edges DeepSeek

"Following the success of our Tülu 3 release in November, we are thrilled to announce the launch of Tülu 3 405B — The first application of fully open post-training recipes to the largest open-weight models.  "With this release, we demonstrate the scalability and effectiveness of our post-training recipe applied at 405B parameter scale. "Tülu 3 405B achieves competitive or superior performance to both Deepseek v3 and GPT-4o, while surpassing prior open-weight post-trained models of the same size including Llama 3.1 405B Instruct and Nous Hermes 3 405B on many standard benchmarks.  Interestingly, we found that our Reinforcement Learning from Verifiable Rewards (RLVR) framework improved the MATH performance more significantly at a larger scale, i.e., 405B compared to 70B and 8B, similar to the findings in the DeepSeek-R1 report.  Overall, our results show a consistent edge over DeepSeek V3, especially with the inclusion of safety benchmarks.

Logic puzzles AI

[Nouha] Dziri’s team showed that LLMs that have only been trained to predict the next word in a sequence —which is most of them —are fundamentally limited in their ability to solve compositional reasoning tasks .  Other researchers have shown that transformers, the neural network architecture used by most LLMs, have hard mathematical bounds when it comes to solving such problems.  Scientists have had some successes pushing transformers past these limits, but those increasingly look like short-term fixes.  If so, it means there are fundamental computational caps on the abilities of these forms of artificial intelligence —which may mean it’s time to consider other approaches. “The work is really motivated to help the community make this decision about whether transformers are really the architecture we want to embrace for universal learning,” said Andrew Wilson, a machine learning expert at New York University who was not involved with this study.

Fiona Shields

"Who’s afraid of artificial intelligence?   "Probably all of us are a little —but the artists at the centre of this year’s Photo Brussels festival have embraced the technology to bring us an intriguing and, at times, optimistic exploration of one of the most concerning developments of our time. "The ambitious curation by the photography academic Michel Poivert gathers together 17 projects at Brussels’ Hangar gallery.  "Together their creations reveal the visual and intellectual potential, along with the current limits, of this wave of promptography ."

Consent razes a boundary?

Musicians, authors, artists and actors worldwide have expressed alarm about their work being used to train generative AI tools that then churn out similar content based upon it. "We should be clear, when we are talking about data here, we actually mean human creators' life works," says Ed Newton Rex, founder of Fairly Trained, which campaigns for AI firms to respect creators' rights. "This is books, this is articles, this is photos. It's works of art. It's records… The whole point of AI training is to learn how to do something and then do more like that." In 2023 a song featuring AI-generated voices of Canadian singers Drake and The Weeknd went viral on social media before being pulled from streaming platforms because it was not their work and they had not consented to it. It didn't stop the track's creator trying to nominate it for a Grammy award. And even though the artists were fake, it was still wildly popular. "I do not think the u...

University of East Anglia

UK’s University of East Anglia has announced a new one-year postgraduate programme MSc in Artificial Intelligence .  Graduates from any background who have a strong interest in computing and data science can apply for the course.  The programme is designed to equip students with both the theoretical knowledge and practical skills required to excel in the rapidly evolving AI landscape.  The curriculum is designed to guide students from foundational programming languages like Python to advanced concepts in AI, ensuring they are industry-ready by the end of the course. It is a full-time study option, incorporating immersive and comprehensive learning experience. 

Mistral Small 3

Mistral AI, the rapidly ascending European artificial intelligence startup, unveiled a new language model today that it claims matches the performance of models three times its size while dramatically reducing computing costs —a development that could reshape the economics of advanced AI deployment. The new model, called Mistral Small 3, has 24 billion parameters and achieves 81% accuracy on standard benchmarks while processing 150 tokens per second.  The company is releasing it under the permissive Apache 2.0 license, allowing businesses to freely modify and deploy it. “We believe it is the best model among all models of less than 70 billion parameters,” said Guillaume Lample, Mistral’s chief science officer, in an exclusive interview with VentureBeat . “We estimate that it’s basically on par with the Meta’s Llama 3.3 70B that was released a couple months ago, which is a model three times larger.”

Winston Cho

"The Motion Picture Association has said that the fact some creators produced parts of a movie with the help of AI shouldn’t render those portions uncopyrightable .  "Take a hypothetical example of a superhero movie. The movie may be copyrighted, but what about scenes involving AI-assisted visual effects depicting a space battle?  "At the forefront of studios’ concerns: The protection of their rights if the underlying characters and scripts are protectable but not certain portions that incorporate material generated by AI.  "A copyright carveout for certain scenes, the MPA has said, isn’t tenable. It’s argued in favor of not having to comply with the copyright office’s registration requirements since applying them would have significant, negative real-world consequences. "

The Authors Guild has finger in the dike

The Authors Guild —one of the largest associations of writers in the US —has launched a new project that allows authors to certify that their book was written by a human, and not generated by artificial intelligence. The Guild says its “Human Authored” certification aims to make it easier for writers to “distinguish their work in increasingly AI-saturated markets,” and that readers have a right to know who (or what) created the books they read.  Human Authored certifications will be listed in a public database that anyone can access.  The project was first announced back in October in response to a deluge of AI-generated books flooding online marketplaces like Amazon and its Kindle ebook platform.

More training data for the win

Image
 

LLMs' maths fragility

"We conduct a large-scale study on several state-of-the-art open and closed models. To overcome the limitations of existing evaluations, we introduce GSM-Symbolic, an improved benchmark created from symbolic templates that allow for the generation of a diverse set of questions. "GSM-Symbolic enables more controllable evaluations, providing key insights and more reliable metrics for measuring the reasoning capabilities of models. "Our findings reveal that LLMs exhibit noticeable variance when responding to different instantiations of the same question.  "Specifically, the performance of all models declines when only the numerical values in the question are altered in the GSM-Symbolic benchmark.  "Furthermore, we investigate the fragility of mathematical reasoning in these models and demonstrate that their performance significantly deteriorates as the number of clauses in a question increases.  "We hypothesize that this decline is due to the fact that curre...

California attorney general’s office demands answers from OpenAI🦹‍♂️

As part of what it described as an ongoing investigation, the California attorney general’s office has sought answers from OpenAI about its reported plan to convert to a for-profit corporation and how it intends to transfer assets out of its existing nonprofit. In a letter sent to the ChatGPT maker Dec. 6, deputy attorney general Christopher Lamerdin cited clauses in OpenAI’s articles of incorporation under which OpenAI’s assets are irrevocably dedicated to its charitable purpose ,  as Lamerdin put it, as well as the office’s responsibility to protect assets held in charitable trust .   In addition to asking about asset transfers, it sought information on OpenAI’s restructuring plan and the value of its assets. The attorney’s general’s office told CalMatters in an email, “The Department of Justice is committed to protecting charitable assets for their intended purpose and takes this responsibility seriously.”

Training data FTW

Maybe it’s com­plete gib­ber­ish, a high-con­cept prac­ti­cal joke con­coct­ed by 15th cen­tu­ry scribes to troll us in the future, know­ing we’d fill in the space of not-know­ing with the most fan­tas­ti­cal­ly strange spec­u­la­tions .  This is a propo­si­tion Stephen Bax , anoth­er con­tender for a Voyn­ich solu­tion, finds hard­ly cred­i­ble. “Why on earth would any­one waste their time cre­at­ing a hoax of this kind?” he asks.  Maybe it’s a rel­ic from an insu­lar com­mu­ni­ty of magi­cians who left no oth­er trace of them­selves.  Sure­ly in the last 300 years every pos­si­ble the­o­ry has been sug­gest­ed, dis­card­ed, then picked up again. Should you care to take a crack at sleuthing out the Voyn­ich mystery —or just to browse through it for curiosity’s sake —you can find the man­u­script scanned at Yale’s Bei­necke Rare Book & Man­u­script Library , which hous­es the vel­lum orig­i­nal.  Or flip through the Inter­net Archive’s dig­i­tal ver­sion…  A...

Mistral AI struggling

France's Mistral AI is facing mounting pressure over its future as an independent European AI champion, as competition intensifies from U.S. tech giants and China's emerging players.   The Paris-based startup, valued at $6.5 billion and backed by Microsoft and Nvidia, has struggled to keep pace with larger rivals despite delivering advanced AI models with a fraction of their resources. The pressure increased this week after China's DeepSeek released a cutting-edge open-source model that challenged Mistral's efficiency-focused strategy.  Mistral CEO Arthur Mensch dismissed speculation about selling to Big Tech companies, saying the firm hopes to go public eventually. However, one investor told the Financial Times that "They need to sell themselves." The stakes are high for Europe's tech ambitions.  Mistral remains the region's only significant player in large language models, the technology behind ChatGPT, after Germany's Aleph Alpha pivoted away ...

9to5Mac on privacy and DeepSeek

"The same privacy concerns apply to most generative AI services.   "Given their very similar privacy policies, it seems the only difference here is that US companies are trusted more than Chinese ones.  "The best advice is never to include personal data in your chatbot requests. "Apple Intelligence is a notable exception to this: the service does not use your data for training purposes.  "Additionally, Apple’s deal with OpenAI means that ChatGPT is also banned from doing so when you access it as a fallback to Apple Intelligence.  "For this reason, accessing ChatGPT via Siri is the safest way to use it."

Analysts debate export-control strategy

Analysts debated the efficacy and relevance of export controls on China’s AI advances and how DeepSeek’s breakthrough will affect future U.S. strategy :  Lennart Heim and Sihao Huang wrote in Heims’s AI governance blog about the ways in which the coverage of DeepSeek’s achievements overlooks the complexity of (and U.S. strategic advantages in) compute access, export controls, and AI development.  Dean W. Ball argued at Lawfare that DeepSeek’s AI breakthrough does not invalidate the U.S.’s basic export control strategy, since the latter is designed more to deny China an AI computing ecosystem rather than prevent the emergence of a single, high-performing Chinese model.  Paul Triolo suggested in his AI Stack Decrypted Substack that the U.S. and China should increase collaboration on AI safety now that advanced AI technology is more easily available to malicious actors, and he challenged the common assumption that attaining nuanced U.S. export controls is even feasible, l...

Chance, not chants 💫

Although meteorites carry the molecules necessary for life, Pizzarello points out that we still don’t know whether they’re responsible for life on Earth or whether they’re the only cosmic vehicle that brought chiral molecules here .  Definitive answers will be difficult to obtain, but Cockell believes scientists can chip away at the puzzle by studying how molecules crucial to life form in space and what processes select for chirality. “What other prebiotic secrets are locked up in unknown compounds [in space]?” he wonders. Regardless of the answer, the study suggests “very deep roots for the chemical evolution that preceded life.”  The idea that chirality, a predisposition inherent in all life as we know it, either began with or arose from the birth of the Milky Way Galaxy some 13.6 billion years ago delivers a dose of cosmic perspective. As Carl Sagan said, “we’re made of star stuff.” The findings also hint at the even bigger question of determinism, which is whether physical...

Not satire

Declassified World War II-era government guide to “simple sabotage” is currently one of the most popular open source books on the internet .  The book, called “Simple Sabotage Field Manual,” was declassified in 2008 by the CIA and “Describes ways to train normal people to be purposefully annoying telephone operators, dysfunctional [rail] train conductors, befuddling middle managers, blundering factory workers, unruly movie theater patrons, and so on. In other words, teaching people to do their jobs badly.”  Over the last week, the guide has surged to become the 5th-most-accessed book on Project Gutenberg , an open source repository of free and public domain ebooks.  It is also the fifth most popular ebook on the site over the last 30 days, having been accessed nearly 60,000 times over the last month (just behind Romeo and Juliet ). 

Satire: Elmo broke the hotline

Image

Qwen2.5-Max ✨

Alibaba Cloud unveiled its Qwen2.5-Max model today, marking the second major artificial intelligence breakthrough from China in less than a week, further rattling U.S. technology markets and intensifying concerns about America’s eroding AI leadership. The new model outperforms DeepSeek’s R1 model —whose success sent Nvidia’s stock plunging 17% on Monday —in several key benchmarks including Arena-Hard, LiveBench and LiveCodeBench.  Qwen2.5-Max also demonstrates competitive results against industry leaders like GPT-4o and Claude-3.5-Sonnet in tests of advanced reasoning and knowledge.

Online sources

Image

DeepSeek distills Dark Lord? 🦹‍♂️

According to a new article by the Financial Times , OpenAI claims to have evidence that DeepSeek, the Chinese startup that has thrown the US tech market into financial turmoil, used the company's proprietary models to train its own open-source LLM, called R1.  This would represent a potential breach of intellectual property, as it goes against the OpenAI terms of service agreement. In the article the FT writes that a source at OpenAI claims it has evidence of “distillation” occurring, which is a technique used by developers to leapfrog on the work done by larger models to achieve similar results at a much lower cost. The OpenAI terms of service clearly state that users cannot copy any of its service or “use output to develop models that compete with OpenAI.” David Sacks, the Whitehouse crypto and AI czar said in an interview on Fox that there is “substantial evidence” of distillation occurring from DeepSeek.

Satire by The Beaverton🫥

Satire by The Beaverton : “DeepBean’s new beans only require a third of the water that American beans need to not grow into anything of value,” an analyst said. “Any serious investor or cow owner would do well to give DeepBean a look.” American magic bean companies like Beanco, The Boston Bean Company, and Nvidia have already shed hundreds of billions of dollars in stock value as investors contemplate getting their magic beans from overseas. “Society can’t advance without magic beans to grow into bizarre and ultimately worthless simulacrums of real beanstalks,” one investor said. “If we’re only wasting moderate amounts of water on them instead of massive amounts, that’s a plus.” While some experts believe a Western bean bubble is now bursting, others see this as a sign to recommit and prevent a disastrous bean gap.

It's Going Down

"Beyond the increasing financial and environmental costs, accelerated AI production also means creating technologies that by the admission of their own creators, will automate out of existence many jobs —and not just white-collar ones.   "Many fast-food chains are already working to automate out their workforce through AI, from drive through windows to inside the restaurants themselves.  "This reality creates a paradox: Trump barely squeaked out a win in 2024 through weaponizing growing resentment against neoliberalism; an economic system defined by corporate globalization and a declining standard of living.  "But as Forbes wrote, '[A]utomation technology has been the primary driver of U.S. income inequality over the past 40 years…50% to 70% of changes in U.S. wages since 1980 can be attributed to wage declines among blue-collar workers replaced or degraded by automation.'  "The push by Trump to fuel the growth of AI will of course only accelerate thi...

Faisal Islam: Wake-up Call

If the US Congress was sufficiently concerned about TikTok to ban it, then surely a table-topping AI program could be highly problematic. President Trump's argument this morning was that DeepSeek's innovation was "positive" and "a wake-up call". China has not been prominent as the first target of Trump tariffs. There is still an obvious balancing act for the UK government here. But this sort of innovation and its impact on the world was exactly why the chancellor visited Beijing a fortnight ago. She said at the time she wanted a long-term relationship with China that is "squarely in our national interest" with the visit part of a "commitment to explore deeper economic co-operation" between Prime Minister Sir Keir Starmer and President Xi. Other European nations such as Spain have encouraged China not just to set up factories but to transfer its advanced battery technology, for example, into Europe.

Stay Kallm

[Pat] Gelsinger wrote that DeepSeek should remind the tech industry of its three most important lessons:  Lower costs mean wider-spread adoption;  Ingenuity flourishes under constraints; and  Open wins.  "DeepSeek will help reset the increasingly closed world of foundational AI model work," he wrote .  OpenAI and Anthropic are both closed source. Gelsinger told TechCrunch that R1 is so impressive, Gloo has already decided not to adopt and pay for OpenAI.  Gloo is building an AI service called Kallm , which will offer a chatbot and other services. "My glue engineers are running R1 today," he [Gelsinger] said. "They could’ve run o1 —well, they can only access o1, through the APIs."

Jensen Huang's net worth declined roughly $21 billion

Nvidia lost close to $600 billion in market cap on Monday, the biggest drop for any company on a single day in U.S. history. The chipmaker's stock price plummeted 17% to close at $118.58. It was Nvidia's worst day on the market since March 16, 2020, which was early in the Covid pandemic: After Nvidia surpassed Apple last week to become the most valuable publicly traded company, the stock's drop Monday led a 3.1% slide in the tech-heavy Nasdaq. The sell-off was sparked by concerns that Chinese artificial intelligence lab DeepSeek is presenting increased competition in the global AI battle.  In late December, DeepSeek unveiled a free, open-source large language model that it said took only two months and less than $6 million to build, using reduced-capability chips from Nvidia called H800s. 

Frustrated AI can't talk, won't talk

Image

Also the source code is public but whatever

Image

Suzanne Nossel

"In China, where all internet companies operate at the pleasure of the Chinese Communist Party, authorities have been systematically purging vast swaths of the internet. A 2024 post on WeChat revealed that nearly all information posted online between late 1990s and mid-2000s had been expunged. The post itself was soon censored and vanished . "Events like the Sichuan earthquake of 2008, which killed more than 68,000 people and prompted a ferocious online debate over China’s shoddy building standards, have been almost entirely scrubbed from China’s online realm. Recent scandals, including one involving the transportation of cooking oil in unsanitary tankers, are also routinely suppressed. Discussion of the 1989 Tiananmen Square uprising is still considered so taboo that many young Chinese are unaware until they travel overseas and have unfettered access to the internet and history books. "Intensifying censorship under Chinese President Xi Jinping’s rule over the last 12 ye...

Benchmarks for DeepSeek

Image

DeepSeek-V3 Technical Report

"We present DeepSeek-V3, a strong Mixture-of-Experts (MoE) language model with 671B total parameters with 37B activated for each token .  "To achieve efficient inference and cost-effective training, DeepSeek-V3 adopts Multi-head Latent Attention (MLA) and DeepSeekMoE architectures, which were thoroughly validated in DeepSeek-V2.  "Furthermore, DeepSeek-V3 pioneers an auxiliary-loss-free strategy for load balancing and sets a multi-token prediction training objective for stronger performance.  "We pre-train DeepSeek-V3 on 14.8 trillion diverse and high-quality tokens, followed by Supervised Fine-Tuning and Reinforcement Learning stages to fully harness its capabilities. Comprehensive evaluations reveal that DeepSeek-V3 outperforms other open-source models and achieves performance comparable to leading closed-source models.  "Despite its excellent performance, DeepSeek-V3 requires only 2.788M H800 GPU hours for its full training. In addition, its training proces...

Steven Sinofsky

"The big problem we have is that the big scale solutions, no matter all the progress, are consuming too much capital.   "But beyond that the delivery to customers has been on an unsustainable path. It is a path that works against the history of computing, which is that resources needed become free, not more expensive.  "The market for computing simply doesn’t accept solutions that cost more, especially consumption-based pricing. We’ve seen Microsoft and Google do a bit of resetting with respect to pricing in the hopes of turning these massive CapEx efforts into direct revenue.  "I wrote at the time of the initial pricing announcements that there was no way this would be sustainable. It took about a year. Laudable goal for sure but just not how business customers of computing work.  "At the same time, Apple is focused on the 'mostly free' way of doing AI, but the results are at best mixed, and they still have a lot of CapEx going on. "Given that it...

Ben Berkowitz

"Generative AI, as a technology and an industry, has developed so quickly that there hasn't been much time for long-term thinking. "If AI really is that powerful and if the Chinese really have found a way to build it faster and cheaper, [then] any company that's capable of leveraging the technology could win in the long term with more accessible growth opportunities. "'While there is a contention that DeepSeek's efficient training methods could reduce the demand for high-end Nvidia GPUs , potentially affecting sales, it is also plausible that the more cost-effective approach will result in more demand for hardware for those looking to train proprietary models,' Mark Klein, CEO of investment fund SuRo Capital, said in a note Monday." 

How do you really feel?

Image

China's AI space

China’s technology leaders, from Alibaba (BABA) Group Holding Ltd. and Baidu Inc. to Tencent Holdings Ltd., have poured significant money and resources into the race to acquire hardware and customers for their AI ventures .  Alongside Kai-Fu Lee’s 01.AI startup, DeepSeek stands out with its open-source approach —designed to recruit the largest number of users quickly before developing monetization strategies atop that large audience. Because DeepSeek’s models are more affordable, it’s already played a role in helping drive down costs for AI developers in China, where the bigger players have engaged in a price war that’s seen successive waves of price cuts over the past year and a half. 

Stunner

Tech stocks make up a significant chunk of the market —tech constitutes about 45% of the S&P 500, according to Keith Lerner, analyst at Truist. “The bottom line is the US outperformance has been driven by tech and the lead that US companies have in AI,” Lerner said. “The DeepSeek model rollout is leading investors to question the lead that US companies have and how much is being spent and whether that spending will lead to profits (or overspending).” This week kicks off a series of tech companies reporting earnings, so their response to the DeepSeek stunner could lead to tumultuous market movements in the days and weeks to come. But one achievement, albeit a gobsmacking one, may not be enough to counter years of progress in American AI leadership. And a massive customer shift to a Chinese startup is unlikely. So the market selloff may be a bit overdone —or perhaps investors were looking for an excuse to sell. 

DeepSeek surges in the West

The surge in popularity of Chinese artificial intelligence (AI) app DeepSeek has sparked a selloff of shares in major tech companies after it overtook rivals such as ChatGPT to become the top-rated free app on Apple's App Store. Nvidia, Microsoft and Meta were set to open down on Monday and the development knocked European share prices. The app has grown in popularity since its launch, challenging the widely held belief that the US is the unassailable leader in AI and prompting questions about the scale of investments US firms are planning. It is powered by the open-source DeepSeek-V3 model, which its researchers claim was developed for less than $6m —significantly less than the billions spent by rivals. But this claim has been disputed by others in the AI space. DeepSeek's emergence comes as the US is restricting the sale of the advanced chip technology that powers AI to China. 

Wall Street reacts

Founded in 2023 by Liang Wenfeng, the former chief of AI-driven quant hedge fund High-Flyer, DeepSeek’s models are open source and incorporate a reasoning feature that articulates its thinking before providing responses. Wall Street’s reactions have been mixed.  While brokerage firm Jefferies warns that DeepSeek’s efficient approach “ punctures some of the capex euphoria” following recent spending commitments from Meta and Microsoft —each exceeding $60 billion this year —Citi is questioning whether such results were actually achieved without advanced GPUs. Goldman Sachs sees broader implications, suggesting the development could reshape competition between established tech giants and startups by lowering barriers to entry. 

Vatican Commission on Artificial Intelligence

"In the area of security, the Guidelines establish that the use of Artificial Intelligence will be regulated by specific implementing regulation. Thus, a set of special regulation will dictate its use in the field of protection and security. "In this regard, the Guidelines provide for the establishment of a Commission on Artificial Intelligence, to be appointed by the President of the Governorate.  "It will be composed of five members, including officials from the Legal Office, and from the Directorates of Telecommunications and Information Systems, and of Security and Civil Protection Services. "The Commission’s main responsibilities will include  Drafting laws and implementing regulations for the use of Artificial Intelligence,  Providing opinions on proposals for the testing and implementation of systems within the Vatican,  Reporting potential risks to the Governorate and  Drafting a semi-annual report on the impact of the use of Artificial Intelligence wit...

Class clown 🫥

AI technology is now a major part of most industries, including business, entertainment, and law, so it’s not surprising that it’s also becoming a tool in the teacher’s toolbox. Local media reported this month that in Arizona, students at a virtual academy will be taught by AI for two hours each day.  In London, high school students prepared for exams with personalized learning using AI, which replaced their teachers. “Students will benefit enormously from AI-powered adaptive learning, which allows every student to learn at their own pace rather than having to keep pace with a class, which often progresses too quickly for some students and too slowly for others,” a coprincipal from the David Game College told BI in August.

Code reduces energy consumption

Through their research, Martin Karsten, professor of computer science in the University of Waterloo’s Math Faculty, and computer science grad student Peter Cai, discovered that the way that data centers were processing packets of network traffic was inefficient, and devised a small change to make it far more efficient. “We didn’t add anything,” Karsten said. “We just rearranged what is done when, which leads to a much better usage of the data center’s CPU caches . It’s kind of like rearranging the pipeline at a manufacturing plant, so that you don’t have people running around all the time.” Karsten teamed up with Joe Damato, distinguished engineer at content delivery network firm Fastly, to develop a small section of code —approximately 30 lines —that would improve Linux’s network traffic processing. If adopted, the new method could reduce the energy consumption of important data center operations by as much as 30 percent , the researchers said. The team tested their solution’s effect...

Data centers can deepen America’s reliance on fossil fuels

High demand for electricity from U.S. data centers is delaying the planned closures of fossil fuel power plants.  At least 17 fossil fuel generating units at seven power plants totaling 9,100 megawatts (MW) of capacity have been delayed or are at risk of being delayed due to concerns about rising electricity demand, with data centers named as a key concern in many cases. Utilities are also emphasizing concerns about computing energy demand to propose investments in new fossil fuel power plants.  At least 10,808 MW of new fossil-fuel generation is being planned to meet projected demand, including, in many cases, from data centers. Much of the increased demand for energy from data centers is attributable to new energy-intensive computing practices such as artificial intelligence (AI) and cryptocurrency mining.

Will Hutton

"US 'free speech' tech giants will unleash a torrent of disinformation that grossly disfigures our understanding of reality, and indulge a deluge of online mayhem that provokes sexual abuse and feeds violence.  "There will be no check on biased AI algorithms used to guide everything from court judgments to recommendations on staff hiring.  "Hacking will explode —and employers will use AI to monitor our every second at work.  "There are other more existential dangers from Trump’s unilateral recklessness —there could, for example, be some  AI miscalculation over gene-editing. Worse,  AI-driven drones will kill indiscriminately from the air against all the rules of war. Would  AI-controlled nuclear weapons be failsafe?  "Few believed, including until recently Elon Musk, that the US’s leading AI companies had the processes to safely manage the ever smarter machine-generated intelligence they were spawning ." 

Yejin Choi

"Collaborating with moral philosophers like John Tasioulas at University of Oxford on AI’s moral decision making sparked my interest in exploring how large language models (LLMs) might make moral decisions.  "This led me to investigate pluralistic alignment —the concept that there could be multiple answers to a question rather than a single 'gold' answer.  "Recent AI models often operate under the assumption of a gold answer, but reality is far more complex and influenced by factors like cultural norms.  "This realization underscored the importance of ensuring that AI is truly safe to humans.  "We must ensure that AI is not narrowly optimized for a single outcome, and I am eager to heavily invest in this work at Stanford HAI."

Interview with Dario Amodei

Image

DeepSeek rules

DeepSeek, the Chinese artificial intelligence (AI) lab behind the innovation, unveiled its free large language model (LLM) DeepSeek-V3 in late December 2024 and claims it was built in two months for just $5.58 million —a fraction of the time and cost required by its Silicon Valley competitors.  Following hot on its heels is an even newer model called DeepSeek-R1, released Monday (Jan. 20) .  In third-party benchmark tests, DeepSeek-V3 matched the capabilities of OpenAI's GPT-4o and Anthropic's Claude Sonnet 3.5 while outperforming others, such as Meta's Llama 3.1 and Alibaba's Qwen2.5, in tasks that included problem-solving, coding and math. Now, R1 has also surpassed ChatGPT's latest o1 model in many of the same tests.  This impressive performance at a  Fraction of the cost of other models, its  Semi-open-source nature, and its  Training on significantly less graphics processing units (GPUs)  has wowed AI experts and raised the specter of China's AI ...

Quantum consciousness 💫

Image

Larry says, "Citizens will be on their best behavior."

Ellison said AI would be used in the future to constantly watch and analyze vast surveillance systems, like security cameras, police body cameras, doorbell cameras, and vehicle dashboard cameras. "We're going to have supervision," Ellison said. "Every police officer is going to be supervised at all times, and if there's a problem, AI will report that problem and report it to the appropriate person. Citizens will be on their best behavior because we are constantly recording and reporting everything that's going on." Ellison also expects AI drones to replace police cars in high-speed chases . "You just have a drone follow the car," Ellison said. "It's very simple in the age of autonomous drones."  He did not say if those drones would broadcast the chases on network news. Ellison's company, Oracle, like almost every company these days, is aggressively pursuing opportunities in the AI industry. It already has several projects in t...

Alongside AI: Experimentation

" Pasta alla Cacio e pepe  is a traditional Italian dish made with pasta, pecorino cheese, and pepper. Despite its simple ingredient list, achieving the perfect texture and creaminess of the sauce can be challenging .  "In this study, we systematically explore the phase behavior of Cacio and pepe sauce, focusing on its stability at increasing temperatures for various proportions of cheese, water, and starch.  "We identify starch concentration as the key factor influencing sauce stability, with direct implications for practical cooking. Specifically, we delineate a regime where starch concentrations below 1% (relative to cheese mass) lead to the formation of system-wide clumps, a condition determining what we term the Mozzarella Phase  and corresponding to an unpleasant and separated sauce.  "Additionally, we examine the impact of cheese concentration relative to water at a fixed starch level, observing a lower critical solution temperature that we theoretically ...

Concept and location neurons in the human brain provide the ‘what’ and ‘where’ in memory formation

"Our brains create new memories by capturing the ‘who/what’, ‘where’ and ‘when’ of everyday experiences.  "On a neuronal level, mechanisms facilitating a successful transfer into episodic memory are still unclear.  "We investigated this by measuring single neuron activity in the human medial temporal lobe during encoding of item-location associations .  "While previous research has found predictive effects in population activity in human MTL structures, we could attribute such effects to two specialized sub-groups of neurons: concept cells in the hippocampus, amygdala and entorhinal cortex (EC), and a second group of parahippocampal location-selective neurons.  "In both item- and location-selective populations, firing rates were significantly higher during successfully encoded trials.  "These findings are in line with theories of hippocampal indexing, since selective index neurons may act as pointers to neocortical representations.  "Overall, activat...

Nearly Optimal List Labeling: See-Saw Algorithm

"The list-labeling problem captures the basic task of storing a dynamically changing set of up to n elements in sorted order in an array of size m = ( 1 + Θ ( 1 ) ) n .  "The goal is to support insertions and deletions while moving around elements within the array as little as possible. "Until recently, the best known upper bound stood at O ( log 2 n ) amortized cost. This bound, which was first established in 1981, was finally improved two years ago, when a randomized O ( log 3 / 2 n ) expected-cost algorithm was discovered.  "The best randomized lower bound for this problem remains Ω ( log n ) , and closing this gap is considered to be a major open problem in data structures. "In this paper, we present the See-Saw Algorithm, a randomized list-labeling solution that achieves a nearly optimal bound of O ( log n polyloglog n ) amortized expected cost.  "This bound is achieved despite at least three lower bounds showing that this type of result is imp...

High-Temperature Gibbs States are Unentangled and Efficiently Preparable

"We show that thermal states of local Hamiltonians are separable above a constant temperature.   "Specifically, for a local Hamiltonian H on a graph with degree d , its Gibbs state at inverse temperature β , denoted by ρ = e − β H / tr ( e − β H ) , is a classical distribution over product states for all β < 1 / ( c d ) , where c is a constant.  "This sudden death of thermal entanglement upends conventional wisdom about the presence of short-range quantum correlations in Gibbs states. "Moreover, we show that we can efficiently sample from the distribution over product states. In particular, for any β < 1 / ( c d 3 ) , we can prepare a state ϵ -close to ρ in trace distance with a depth-one quantum circuit and poly ( n ) log ( 1 / ϵ ) classical overhead.  "A priori the task of preparing a Gibbs state is a natural candidate for achieving super-polynomial quantum speedups , but our results rule out this possibility above a fixed constant temperatu...

Awwwww 🫥

"Contrary to expectations revealed in four surveys, cross country data and six additional studies find that people with lower AI literacy are typically more receptive to AI .  "This lower literacy-greater receptivity link is not explained by differences in perceptions of AI’s capability, ethicality, or feared impact on humanity. Instead, this link occurs because people with lower AI literacy are more likely to perceive AI as magical and experience feelings of awe in the face of AI’s execution of tasks that seem to require uniquely human attributes.  "In line with this theorizing, the lower literacy-higher receptivity link is mediated by perceptions of AI as magical and is moderated among tasks not assumed to require distinctly human attributes.  "These findings suggest that companies may benefit from shifting their marketing efforts and product development towards consumers with lower AI literacy.  "Additionally, efforts to demystify AI may inadvertently reduc...

ESM3 ✨

"More than three billion years of evolution have produced an image of biology encoded into the space of natural proteins.  "Here we show that language models trained at scale on evolutionary data can generate functional proteins that are far away from known proteins .  "We present ESM3, a frontier multimodal generative language model that reasons over the sequence, structure, and function of proteins.  "ESM3 can follow complex prompts combining its modalities and is highly responsive to alignment to improve its fidelity.  "We have prompted ESM3 to generate fluorescent proteins. Among the generations that we synthesized, we found a bright fluorescent protein at a far distance (58% sequence identity) from known fluorescent proteins, which we estimate is equivalent to simulating five hundred million years of evolution." 

West Greenland

"We analyzed a unique, long-term lake dataset and found that compound climate extremes pushed Arctic lakes across a tipping point.  "As terrestrial–aquatic linkages were strengthened, lakes synchronously transformed from 'blue' lakes with high transparency and low pelagic primary production to 'brown' in less than a year, owing to a large influx of dissolved organic material and metals, with iron concentrations increasing by more than two orders of magnitude .  "The browning of lake waters reduced light penetration by 50% across lakes. The resulting light limitation altered plankton distributions and community structure, including a major reduction in prokaryotic diversity and an increase in algal groups capable of metabolizing organic carbon sources.  "As a result, lakes shifted from being summer carbon sinks to sources, with a >350% increase in carbon dioxide flux from lakes to the atmosphere.  "The remarkably rapid, coherent transformatio...

Interlune Helium-3 for quantum computing

[Interlune CEO Rob] Meyerson explained that Helium-3 could also fuel nuclear fusion reactors on Earth [and that is] a use that [astronaut Jack] Schmitt has championed in the past though investors have shown little interest [and prioritize] other applications such as quantum computing that would yield faster returns . Interlune is sharply focused near-term on extracting Helium-3 for superconducting quantum computing applications.   “We have ways we can go meet that need in the future,” Meyerson said. “Quantum computing is the key demand generator for us.” Meyerson said that Helium-3 now commands a stable price of around $20 million per kilogram.

AI, AI, EO

President Donald Trump signed an executive order on artificial intelligence Thursday that will revoke past government policies his order says “act as barriers to American AI innovation.” To maintain global leadership in AI technology, “we must develop AI systems that are free from ideological bias or engineered social agendas,” Trump’s order says. The new order doesn’t name which existing policies are hindering AI development but sets out to track down and review “all policies, directives, regulations, orders, and other actions taken” as a result of former President Joe Biden’s sweeping AI executive order of 2023, which Trump rescinded Monday.  Any of those Biden-era actions must be suspended if they don’t fit Trump’s new directive that AI should “promote human flourishing, economic competitiveness, and national security.” Trump’s order also calls for the development of an AI action plan within 180 days. Leading the work will be a small group of White House tech and science offici...

Cool Operator 🦹‍♂️

"Today we’re releasing Operator, an agent that can go to the web to perform tasks for you.   "Using its own browser, it can look at a webpage and interact with it by typing, clicking, and scrolling. It is currently a research preview, meaning it has limitations and will evolve based on user feedback.  "Operator is one of our first agents, which are AIs capable of doing work for you independently — you give it a task and it will execute it. "Operator can be asked to handle a wide variety of repetitive browser tasks such as  Filling out forms,  Ordering groceries, and even  Creating memes.  "The ability to use the same interfaces and tools that humans interact with on a daily basis broadens the utility of AI, helping people save time on everyday tasks while opening up new engagement opportunities for businesses."   

Nepenthes

A pseudonymous coder has created and released an open source tar pit to indefinitely trap AI training web crawlers in an infinitely, randomly-generating series of pages to waste their [crawlers] time and computing power.  The program, called Nepenthes after the genus of carnivorous pitcher plants which trap and consume their prey, can be deployed by webpage owners to protect their own content from being scraped or can be deployed offensively as a honeypot trap to waste AI companies’ resources. “It's less like flypaper and more an infinite maze holding a minotaur, except the crawler is the minotaur that cannot get out. The typical web crawler doesn't appear to have a lot of logic. It downloads a URL, and if it sees links to other URLs, it downloads those too,” Aaron B, the creator of Nepenthes, told  404 Media .  “Nepenthes generates random links that always point back to itself —the crawler downloads those new links. Nepenthes happily just returns more and more lists of...

Kerem Gülen on Galaxy S25 ✨

"Samsung’s new Galaxy S25 phones are all about AI.   "What you need to know, in summary is this:  "Do more with less effort: You can tell your phone to do stuff across different apps, like searching your email or adding events to your calendar. No more jumping between apps, your phone just takes care of it.  "Find things faster: Looking for a specific photo or setting? Just ask your phone. It’s like having a built-in search engine for everything on your device.  "Sound control in videos: They’ve got this Audio Eraser thing that lets you control the different sounds in a video. So you can, like, mute the background noise and make the voices clearer. Pretty cool, huh ? "AI for your photos: Upload a picture and the AI can tell you all about it. It can even find similar photos online. "Easy GIF making: You can make GIFs straight from videos now. No extra apps or anything."

Musk vs Swisher vs Trump

Image

LinkedIn, source for AI training

On January 21st, 2025, a group of LinkedIn Premium users filed a lawsuit against the platform on the grounds that their private data had been shared with third parties without explicit consent .  According to the plaintiffs, LinkedIn quietly implemented a privacy policy update in September 2014, which allowed user data to be utilized for AI training.  However, the users claim that this update was buried in the fine print, making it easy for unsuspecting users to overlook.  The lawsuit accuses LinkedIn of violating User trust, breaching contractual agreements and potentially running afoul of the U.S. data protection laws.  Now if you’ve been on the internet for a while, this isn’t the only company that has mined user data to enhance AI capabilities.  The AI companies themselves, that is to say META, OpenAI, Microsoft to list a few, have been in continuous legal battles regarding how they source their AI training data. 

Anger rampancy 🦹‍♂️

Image

AI has pwn markt?

Some of the market’s most forceful pushes upward came from AI-related companies .  Oracle climbed 6.2% to add to its 7.2% rise the day before, ahead of the expected announcement that came late Tuesday about Stargate , a joint venture the White House said will start building out data centers and the electricity generation needed for the further development of AI in Texas. The partnership formed by Oracle, OpenAI and SoftBank will invest up to $500 billion. SoftBank Group Corp.’s stock in Tokyo rose 10.6%. Other AI-related stocks also climbed, furthering their already fantastic run. Nvidia, the company whose chips are powering much of the move into AI, rose 3.6%. Its stock is above $145 after sitting below $18 just two years ago.

Stargate has cash?

Elon Musk is already casting doubt on OpenAI’s new, up to $500 billion investment deal with SoftBank and Oracle, despite backing from his allies —including President Donald Trump. The companies are teaming up for what they call the Stargate Project ( presumably named after the sci-fi franchise) which aims to spend $500 billion over the next four years building artificial intelligence infrastructure in the U.S. The companies said Tuesday that they would begin deploying $100 billion “immediately,” which will “generate massive economic benefit for the entire world.” As the deal was announced Tuesday in the White House, Trump —standing alongside Oracle Chairman Larry Ellison, OpenAI CEO Sam Altman, and SoftBank CEO Masayoshi Son —said the group would create a “new American company” that would generate more than 100,000 U.S. jobs.  Ellison said the group’s first project, a one-million-square-foot data project, is already underway in Texas. There’s just one problem with Stargate, accordi...