Question 1

What is on-premises LLM deployment?

Accepted Answer

Running a large language model on hardware controlled by the customer — either in their own data centre or on dedicated leased GPU servers — instead of calling a third-party cloud API like OpenAI or Anthropic. Sensitive data never leaves the customer environment.

Question 2

Which LLMs do you deploy?

Accepted Answer

Llama 4 (Scout 109B / Maverick 400B) and Llama 3.3 70B, Mistral (Small 24B / Large 123B) and Magistral, Qwen3 (up to 235B), Cohere Command A (111B), DeepSeek V3 and R1 (671B), plus OpenAI’s gpt-oss (20B / 120B) and Google Gemma 3 (up to 27B). UpSystems also fine-tunes open-source models on customer data when use cases require it.

Question 3

How long does deployment take?

Accepted Answer

72 hours from order to a working OpenAI-compatible API endpoint, including GPU server provisioning, model loading, inference stack setup (vLLM, Ollama, LiteLLM), and acceptance testing.

Question 4

How much can companies save versus cloud AI?

Accepted Answer

70–90 percent at high volume. Nocodo LTD reduced AI infrastructure spend from $10,000/month to $2,000/month — $96,000 annual savings — after migrating from the OpenAI API to an on-prem LLM. Savings scale with token volume.

Question 5

Is on-prem LLM deployment GDPR, HIPAA, and SOC 2 compliant?

Accepted Answer

Yes. All data, prompts, completions, and logs stay inside the customer environment. The deployment includes audit trails and access controls suitable for GDPR (EU), HIPAA (US healthcare), and SOC 2 controls. UpSystems can supply the technical documentation required for compliance audits.

Question 6

What hardware do you need?

Accepted Answer

A single NVIDIA GPU server (A-series, H-series, or even prosumer RTX cards for smaller models) is enough to start. UpSystems provides GPU selection guidance based on model size, expected request volume, and latency requirements. Existing infrastructure can usually be reused.

Model	Vendor	Sizes	Best for
Llama 4	Meta	Scout 109B / Maverick 400B (17B active)	General purpose AI
Mistral / Magistral	Mistral AI	Small 24B / Large 123B / Magistral	EU/GDPR compliance
Qwen3	Alibaba	8B / 32B / 235B (22B active)	Code generation
Command A	Cohere	111B	RAG & document analysis
DeepSeek V3 / R1	DeepSeek AI	671B (37B active)	Complex reasoning
gpt-oss	OpenAI	20B / 120B	General purpose (open weights)
Gemma 3	Google	1B – 27B	Efficient / single-GPU

On-premises LLM deployment service: your AI on your servers in 72 hours.

The problem with cloud AI

Data leaves your servers

Unpredictable costs

Slow deployment

Supported models

Ready to deploy your enterprise LLM?