Have large language models reached their limit?
In this InTechnology video, Camille talks with Thumas Dullien, aka Halvar Flake. They get into cybersecurity and software optimization through AI and LLMs, artificial general intelligence and other technological advancements, trends in data access and large AI model development, and more.
Cybersecurity and Software Optimization through AI and LLMs
In a previous discussion, Thomas shared insights on AI and machine learning’s effectiveness in stable data distributions versus their reduced efficacy in rapidly changing or maliciously altered distributions. He outlined several uses for large language models (LLMs), particularly in cybersecurity. For defensive strategies, he suggested using AI to generate “garbage data” that appears real but is fake, thereby misleading hackers and safeguarding genuine data. On the offensive front, he mentioned leveraging LLMs for enhanced penetration testing support. In performance optimization, Thomas sees vast potential, citing opportunities in areas like LLMs, AlphaZero-like algorithms, and reinforcement learning.
Artificial General Intelligence (AGI) and Technological Advancements
Addressing the timeline for AGI, Thomas opined that it remains a distant goal. He acknowledged the rapid advancements in image and language generation, which might lead some to anticipate AGI’s earlier arrival. However, he warns of a potential stagnation in LLMs’ capabilities, drawing a parallel to the development plateau experienced by autonomous vehicles. Thomas highlighted the misconception that rapid initial progress in technology would continue indefinitely, not accounting for subsequent slowdowns.
Trends in Data Access and Large AI Model Development
Thomas traced the evolution of computing from centralized systems in the 1950s and 60s, to the decentralized era marked by PCs and the internet from the 1970s to the 90s, and back to centralization with the rise of cloud computing. He pointed out the challenges in centralized AI and cloud computing, noting that currently, only major players can access the vast data needed for large AI models. Thomas expressed hope for a future with less centralized, more accessible, and efficient computing, expanding benefits to a broader audience. He emphasized the need for more efficient models to process and utilize larger data sets at greater speeds. While synthetic data might be useful in some areas, Thomas remarked that its application, especially in language modeling, poses unique challenges.
Thomas Dullien — Mathematician, Cybersecurity, and Software Optimization Expert
Thomas Dullien, recognized under his pseudonym Halvar Flake for pioneering work in reverse engineering and identifying software security flaws, currently serves as a Distinguished Software Engineer at Elastic. Previously, Thomas co-founded optimyze and held the position of CEO until its acquisition by Elastic in 2021. He has had two tenures at Google, first as a Staff Engineer from 2011 to 2015 and then from 2016 to 2018. Thomas is the founder of zynamics GmbH, where he was also the CEO and Head of Research. Google acquired zynamics in 2011. He holds a Master’s degree in Mathematics from Ruhr University Bochum and commenced a Ph.D. program there, which he later left to concentrate on zynamics. Thomas is notably recognized for his contributions to research on Rowhammer.
Check it out. For more information, previous podcasts, and full versions, visit our homepage.
To read more about cybersecurity topics, visit our blog.
#cybersecurity #softwareoptimization #rowhammer
The views and opinions expressed are those of the guests and author and do not necessarily reflect the official policy or position of Intel Corporation.
If you are interested in emerging threats, new technologies, or best tips and practices in cybersecurity, please follow the InTechnology podcast on your favorite podcast platforms: Apple Podcast and Spotify.