Josh Engels
Howdy!
I am currently a second year EECS PhD student at MIT, where I am grateful to be advised by Max Tegmark. Broadly, I am interested in understanding neural networks from the ground up through the lens of interpretability, with an eventual goal of using these insights to ensure powerful AI systems are robust and safe. I am currently supported by the NSF Graduate Research Fellowship Program.Earlier in my PhD, I had the pleasure of working with Julian Shun on building high performance algorithms, with a focus on approximate nearest neighbor search. I also previously spent two enjoyable years working at ThirdAI, where we built a sparse machine learning engine from scratch.
Selected Papers
Decomposing the Dark Matter of Sparse Autoencoders.
Joshua Engels, Logan Smith, and Max Tegmark.
Preprint |
Code |
Twitter
Efficient Dictionary Learning with Switch Sparse Autoencoders.
Anish Mudide, Joshua Engels, Eric J Michaud, Max Tegmark, and Christian Schroeder de Witt.
Preprint |
Code |
Twitter
Not All Language Model Features Are Linear.
Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark.
Preprint |
Code |
Twitter |
Talk
Approximate Nearest Neighbor Search with Window Filters.
Joshua Engels, Benjamin Landrum, Shangdi Yu, Laxman Dhulipala, and Julian Shun.
ICML 2024.
Paper |
Code
DESSERT: An Efficient Algorithm for Vector Set Search with Vector Set Queries.
Joshua Engels, Benjamin Coleman, Vihan Lakshman, and Anshumali Shrivastava
NeurIPS 2023.
Paper |
Code |
Blog Post
Practical Near Neighbor Search via Group Testing.
Joshua Engels*, Benjamin Coleman*, and Anshumali Shrivastava
NeurIPS 2021: Spotlight talk - top 3%
Paper |
Talk |
Code
Other Projects and Writing
SAE Probing: What is it good for? Absolutely something! (2024) - We examine whether SAE probes are more data efficient and robust than activation probes.
Examining the Interaction of Interpretable Features and Training Dynamics in Othello-GPT (2023) - We experiment with promoting linear features while training Othello-GPT.