Andreas Erben

You are not my model anymore - understanding LLM model behavior

Your LLM is a shoggoth with a smiley face mask. Learn what happens when the mask slips and your application breaks.

You are not my model anymore - understanding LLM model behavior
#1about 2 minutes

Unexpected LLM behavior from hidden platform updates

A practical demonstration shows how a cloud provider's content filter update can unexpectedly block access to documents, causing application failures.

#2about 3 minutes

How LLMs generate text and learn behavior

Large language models use a transformer architecture to predict the next token based on probability, with instruction tuning and alignment shaping their final behavior.

#3about 2 minutes

The opaque and complex stack of modern LLM services

Major LLM providers operate in secrecy, and the full technology stack from model weights to the API is complex, leaving developers with limited visibility and control.

#4about 3 minutes

Managing risks from provider filters and short API lifecycles

Cloud provider content filters can change without notice, creating vulnerabilities, while the short lifecycle of model APIs requires constant adaptation.

#5about 4 minutes

Understanding LLMs as alien minds with fragile alignment

LLMs are conceptually like alien intelligences with a fragile, human-like alignment layer that can be bypassed by jailbreaks exploiting internal model circuits.

#6about 2 minutes

How model personalities and behaviors shift between versions

Different LLM versions exhibit distinct behaviors and may ignore system prompts, as shown by a comparison between GPT-4 and a newer reasoning model.

#7about 3 minutes

Using evaluations to systematically test model behavior

Systematically test model behavior using evaluations, which can be automated by generating prompt variations or using pre-built cloud and open-source frameworks.

#8about 4 minutes

Using prompt engineering to mitigate model drift

Mitigate model behavior drift by using advanced prompt engineering techniques like forcing reasoning, providing few-shot examples, and being highly explicit in instructions.

Related jobs
Jobs that call for the skills explored in this talk.

Machine Learning Engineer

Picnic Technologies B.V.

Picnic Technologies B.V.
Amsterdam, Netherlands

Intermediate
Senior
Python
Structured Query Language (SQL)
+1

Matching moments

Prompt injection as an unsolved AI security problem
07:39 MIN

Prompt injection as an unsolved AI security problem

AI in the Open and in Browsers - Tarek Ziadé

The evolving role of the machine learning engineer
02:20 MIN

The evolving role of the machine learning engineer

AI in the Open and in Browsers - Tarek Ziadé

Unlocking LLM potential with creative prompting techniques
04:59 MIN

Unlocking LLM potential with creative prompting techniques

WeAreDevelopers LIVE – Frontend Inspirations, Web Standards and more

Final advice for developers adapting to AI
03:07 MIN

Final advice for developers adapting to AI

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

How AI is changing the freelance developer experience
09:10 MIN

How AI is changing the freelance developer experience

WeAreDevelopers LIVE – AI, Freelancing, Keeping Up with Tech and More

AI lawsuits, code flagging, and self-driving subscriptions
01:02 MIN

AI lawsuits, code flagging, and self-driving subscriptions

Fake or News: Self-Driving Cars on Subscription, Crypto Attacks Rising and Working While You Sleep - Théodore Lefèvre

Building and iterating on an LLM-powered product
05:03 MIN

Building and iterating on an LLM-powered product

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

The security risks of AI-generated code and slopsquatting
05:55 MIN

The security risks of AI-generated code and slopsquatting

Slopquatting, API Keys, Fun with Fonts, Recruiters vs AI and more - The Best of LIVE 2025 - Part 2

Featured Partners

Related Videos

See all videos
Three years of putting LLMs into Software - Lessons learned26:30

Three years of putting LLMs into Software - Lessons learned

Simon A.T. Jiménez

Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails29:00

Beyond the Hype: Building Trustworthy and Reliable LLM Applications with Guardrails

Alex Soto

AI: Superhero or Supervillain? How and Why with Scott Hanselman25:17

AI: Superhero or Supervillain? How and Why with Scott Hanselman

Scott Hanselman

How AI Models Get Smarter35:16

How AI Models Get Smarter

Ankit Patel

Prompt Injection, Poisoning & More: The Dark Side of LLMs23:24

Prompt Injection, Poisoning & More: The Dark Side of LLMs

Keno Dreßel

Inside the Mind of an LLM27:11

Inside the Mind of an LLM

Emanuele Fabbiani

From Traction to Production: Maturing your GenAIOps step by step25:27

From Traction to Production: Maturing your GenAIOps step by step

Maxim Salnikov

Bringing the power of AI to your application.32:26

Bringing the power of AI to your application.

Krzysztof Cieślak

Related Articles

View all articles
DC
Daniel Cranney
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security
Inside last week’s Dev Digest 196 . ⚖️ Political bias in LLMs 🫣 AI written code causes 1 in 5 security breaches 🖼️ Is there a limit to alternative text on images? 📝 CodeWiki - understand code better 🟨 Long tasks in JavaScript 👻 Scare yourself into n...
Dev Digest 196: AI Killed DevOps, LLM Political Bias & AI Security

From learning to earning

Jobs that call for the skills explored in this talk.