Howdy!

I am currently a second year EECS PhD student at MIT, where I am grateful to be advised by Max Tegmark. Broadly, I am interested in understanding neural networks from the ground up through the lens of interpretability, with an eventual goal of using these insights to ensure powerful AI systems are robust and safe. I am currently supported by the NSF Graduate Research Fellowship Program.

Earlier in my PhD, I had the pleasure of working with Julian Shun on building high performance algorithms, with a focus on approximate nearest neighbor search. I also previously spent two enjoyable years working at ThirdAI, where we built a sparse machine learning engine from scratch.

Selected Papers

Decomposing the Dark Matter of Sparse Autoencoders.
Joshua Engels, Logan Smith, and Max Tegmark.
Preprint | Code | Twitter

Efficient Dictionary Learning with Switch Sparse Autoencoders.
Anish Mudide, Joshua Engels, Eric J Michaud, Max Tegmark, and Christian Schroeder de Witt.
Preprint | Code

Not All Language Model Features Are Linear.
Joshua Engels, Eric J. Michaud, Isaac Liao, Wes Gurnee, and Max Tegmark.
Preprint | Code | Twitter

Approximate Nearest Neighbor Search with Window Filters.
Joshua Engels, Benjamin Landrum, Shangdi Yu, Laxman Dhulipala, and Julian Shun.
ICML 2024.
Paper | Code

DESSERT: An Efficient Algorithm for Vector Set Search with Vector Set Queries.
Joshua Engels, Benjamin Coleman, Vihan Lakshman, and Anshumali Shrivastava
NeurIPS 2023.
Paper | Code | Blog Post

Practical Near Neighbor Search via Group Testing.
Joshua Engels*, Benjamin Coleman*, and Anshumali Shrivastava
NeurIPS 2021: Spotlight talk - top 3%
Paper | Talk | Code

* indicates equal contribution

Blog Posts

Bananagrams is NP-complete
Have you ever wondered if Bananagrams is NP-complete? No? Well I think it is, and I’ll prove it!...

What is Set-of-Vector Search?
The set-of-vector search problem is an exciting and unexplored research problem. In our recent paper, we develop DESSERT,...

Simulating Voter Errors with ACT-R
Badly designed ballots may not seem like a pressing problem, but they have already swung multiple elections. For...