HN Reader

Here's the best video I have seen about transformers [1]. It is made by Welch Labs. It talks about DeepSeek and what their main innovation was, but it covers transformers in general too, and I couldn't find any other better description of transformers.

Also, here's an interactive "transformer explainer" that is absolutely mind-blowing [2].

[1] https://www.youtube.com/watch?v=0VLAoVGf_74

[2] https://poloclub.github.io/transformer-explainer/

Honest feedback - I was really excited when I read the opening. However, I did not come away from this without a greater understanding than I already had.

For reference, my initial understanding was somewhat low: basically I know a) what embedding is basically b) transformers work by matrix multiplication, and c) it's something like a multi-threaded Markov chain generator with the benefit of prior-trained embeddings

I love how you represent each token in the form of five stacked boxes, with height, weight etc. depicting different values. Where did you get this amazing idea? I will "steal" it for plotting high dimensionality data.

I'd be surprised if anyone understood transformers from this.

very cool!

I'd also recommend another article on this topic of LLMs discussed a few days ago. I read it to the finish line and understood everything fully:

> How can AI ID a cat?

https://news.ycombinator.com/item?id=44964800

For me, I feel like this could use a little bit more explanation. It's brief and the grammar or cadence is very clunky.

Here is another take on visualizing transformers from Georgia Tech researchers: https://poloclub.github.io/transformer-explainer/

Also, the Illustrated Transformer: https://jalammar.github.io/illustrated-transformer/

Also, this HN comment has numerous resources: https://news.ycombinator.com/item?id=35712334