About
I’m Breno W. Carvalho, a PhD student in Computer Science researching AI interpretability — trying to understand what’s actually happening inside neural networks when they learn.
Currently finishing my PhD while working at a startup building AI tools for lawyers in Brazil. I split my time between reverse-engineering how models compress knowledge and building things that help people work better with AI.
Research
My work focuses on mechanistic interpretability — reverse-engineering the algorithms that neural networks learn. I’m particularly interested in:
- Compression and generalization: How do models decide what to remember vs. what to compress? What drives the transition from memorization to true understanding?
- Grokking: Why do models suddenly “get it” after long periods of perfect training accuracy? What changes internally during this phase transition?
- Weight decay and implicit regularization: How optimization biases shape what models learn and how they represent knowledge.
Recent breakthrough: demonstrated that SGD’s compression bias is strong enough to escape worst-case memorized initializations — models can recover from perfect overfitting and discover generalizing solutions (March 2026 experiments).
Background
Originally from Teresópolis, now based in Rio de Janeiro. Before research, I built things — software, systems, tools that solve real problems. That builder instinct still drives how I approach research: I want to understand AI well enough to build it better.
I live with my partner Matheus and spend free time on gelato science (yes, really), cooking, coffee/latte art, games, and consuming too much pop culture.
What I Write About
This site is where I think out loud about:
- Research findings and experiments
- Technical deep-dives into interpretability
- Building projects and side ventures
- Things I’m learning and figuring out
Academic papers live elsewhere. This is the workshop — sketches, prototypes, honest exploration.
Contact
- GitHub: @brenowca
- Email: [email protected]
- Research updates: mostly through writing here
This site is a work in progress — like research, like me.