HN Reader

Apparently, protecting the API is not planned: https://github.com/ollama/ollama/issues/849

For my own purposes I either restrict ollama's ports in the firewall, or I put some proxy in front of it that blocks access of some header with some predefined api key is not present. Kind of clunky, but it works.

I can think of no reason to be surprised by this, except that Cisco is the one reporting it. That part is surprising.

I’d expect Cisco to publish an article on thousands of Cisco devices with default passwords still there in the open.

Definitely not credible to speak about ML stuff and of course - Ollama has never been production-ready in the sense iOS (Cisco’s) was.

Shodan also has built-in detection for some of them. For example, you can search for "product:ollama" (https://www.shodan.io/search?query=product%3Aollama). Or if you have access to the tag filter then simply "tag:ai" (https://www.shodan.io/search/report?query=tag%3Aai).

Similarly a lot of projects using gradio come with a tunnel/public proxy enabled out of the box. ie instantly publicly accessible just by running it. Behind a long unique uuid looking url which provides some measure of security by obscurity but wow was still surprised first time I saw that.

Must be a good time to be in security space with this sort of stuff plus the inevitable vibe code security carnage

>each identified endpoint is programmatically queried to assess its security posture, with a particular focus on authentication and authorization mechanisms.

I know it's commonplace, but is this unauthorized access in terms of the CMA (UK) or CFAA (USA)?

Why are people running ollama on public servers.

Is this thanks to everyone thinking they can code now and not understanding what they’re doing.

Make it make sense

The article itself appears to be largely AI-edited. And I'm really surprised that anyone would want to write an article on this, I assumed it was widely known? You can go onto Censys and find thousands of exposed instances for lots of self-hostable software, for LLM there are exposed instances of things like kobold, for image gen there's sd-webui, InvokeAI and more.

I understand the concern here but isn't this the same as making any other type of server public? This is just regarding servers hosting LLMs, which I wouldn't even consider a huge security concern vs hosting a should-be-internal tool publicly.

Servers that shouldn't be made public are made public, a cyber tale as old as time.

I'm surprised Shodan is legal. Just because someone made a mistake when setting up their network doesn't mean you're authorized.

The stakes aren’t that high yet for Ollama to warrant cumbersome auth mechanisms.

Ollama has no auth mechanism by default... You have to wonder why they never focused on that

largely the fault of n8n

Cheap (almost free) highly parallel inference. Nice!

Free inference. Yay

how many ppl are using Ollama in production though

Shodan? Like from system shock?

Another great use of a personal VPN - I work at https://www.defined.net (which uses Nebula as the underlying VPN technology) and also personally use our free tier (up to 100 hosts) for everything. Having my Ollama instances available only over my VPN overlay network is very slick.