HN Reader

Methodology is one thing; I can't really agree that deploying an LLM to do sums is great. Almost as hilarious as asking "What's moon plus sun?"

But phenomenon is another thing. Apple's numerical APIs are producing inconsistent results on a minority of devices. This is something worth Apple's attention.

It is a bug in MLX that has been fixed a few days ago: https://github.com/ml-explore/mlx/pull/3083

Low level numerical operation optimizations are often not reproduceable. For example: https://www.intel.com/content/dam/develop/external/us/en/doc... (2013)

But it's still surprising that that LLM doesn't work on iPhone 16 at all. After all LLMs are known for their tolerance to quantization.

I clicked hoping this would be about how old graphing calculators are generally better math companions than a phone.

The best way to do math on my phone I know of is the HP Prime emulator.

The author is assuming Metal is compiled to ANE in MLX. MLX is by-and-large GPU-based and not utilizing ANE, barring some community hacks.

Posting some code that reproduces the bug could help not only Apple but you and others.

I wish he would have tried on a different iPhone 16 Pro Max to see if the defect was specific to that individual device.

Does it bother anyone else that the author drops "MiniMax" there in the article without bothering to explain or footnote what that is? (I could look it up, but I think article authors should call out these things).

Interesting post, but the last bit of logic pointing to the Neural Engine for MLX doesn’t hold up. MLX supports running on CPU, Apple GPU via Metal, and NVIDIA GPU via CUDA: https://github.com/ml-explore/mlx/tree/main/mlx/backend

What expense app are you building? I really want an app that helps me categorize transactions for budgeting purposes. Any recommendations?

I'd think other neural-engine using apps would also have weird behavior. Would've been interesting to try a few App Store apps and see the weird behavior

Maybe this is why my damn keyboard predictive text is so gloriously broken

Good article. Would have liked to see them create a minimal test case, to conclusively show that the results of math operations are actually incorrect.

I love to see real debugging instead of conspiracy theories!

Did you file a radar? (silently laughing while writing this, but maybe there's someone left at Apple who reads those)

>"What is 2+2?" apparently "Applied.....*_dAK[...]" according to my iPhone

At least the machine didn't say it was seven!

The real lesson here isn't even about Apple. It's about debugging culture

So the LLM is working as intended?

> Update on Feb. 1st: > Well, now it's Feb. 1st and I have an iPhone 17 Pro Max to test with and... everything works as expected. So it's pretty safe to say that THAT specific instance of iPhone 16 Pro Max was hardware-defective.

nothing to see here.

My thousand dollar iPhone can't even add a contact from a business card.

I also would like to see if the same error happens in another phone with the exactly same model.

.

neural nets or AI are very bad at math, it can only produce what's in the training data. So if you have trained it from 1+1 to 8+8 it can't do 9+9, it's not like a child brain that it can make logical conclusions.