Why do we see no companies offering to deploy local LLMs?

It seems an obvious idea but we cannot find anyone offering b2b local LLMs setup to corporations that doesn't want to use external ones & keep their data in-house.. ie airlines, governments, attorneys, accountants..

Shameless plug, but super relevant maybe-

I’m a product marketing guy for Nutanix. We have a new LLM deployment solution that can run on any CNCF Kubernetes where you can deploy LLMs from a validated list from Hugging Face or NVIDIA NIM, or upload your own.

You can then create RBAC-controlled endpoints to connect your GenAI apps up to all with a point and click interface.

Check it out with basic walkthrough on the product page: https://www.nutanix.com/products/nutanix-enterprise-ai

6 hours agoby beachhaze

I see large companies leveraging their large technical teams and external consulancies and technology providers to build their own e.g. banks on IBM. And I think for small companies it's impossible to beat the value for money a Saas type solution provides and the big APIs like ChatGPT and Claude. For the companies inbetween I'd imageine the risk to reward of picking a stack might be too high right now given how fast everything is moving. There is probably room for someone to do it next year but I don't get the sense that the market is ready for it just yet.

1 day agoby isherlock

My understanding, and I could be wrong here so I’m happy to be corrected, is that AI companies like OpenAI are perfectly capable of operating an instance of their product that doesn’t send data back to the source.

They discuss this on the Enterprise page:

Protect your data with enterprise-grade privacy, security, and deployment tools You own and control your business data in ChatGPT Enterprise. We do not train on your business data or conversations, and our models don’t learn from your usage. ChatGPT Enterprise is also SOC 2 compliant and all conversations are encrypted in transit and at rest. Our new admin console lets you manage team members easily and offers domain verification, SSO, and usage insights, allowing for large-scale deployment into enterprise. See our privacy page and our Trust Portal (opens in a new window) for more details on how we treat your data.

https://openai.com/index/introducing-chatgpt-enterprise/

So it’s very likely that most companies find this reassuring enough and therefore don’t necessarily care too much about running models locally. Anyone that needs security greater than this probably has the resources to develop AI capabilities in-house.