Sponsored Content

DEV Community

#vllm

Posts

👋 Sign in for the ability to sort posts by relevant, latest, or top.
The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One
xbill profile

The Cheapest CUDA GPU on AWS Has an Arm CPU — and You Probably Want the Intel One

#aws #vllm #cuda #machinelearning
1
Comments
11 min read
vLLM v0.28.0: the breaking change small GPU users must read
induwara_ashinsana_9e4d5b profile

vLLM v0.28.0: the breaking change small GPU users must read

#vllm #llminference #opensource
Comments
5 min read
Installing Rust for vLLM on Graviton: a G5g walk-through 🦀
xbill profile

Installing Rust for vLLM on Graviton: a G5g walk-through 🦀

#rust #vllm #aws #cuda
Comments
10 min read
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
Comments
9 min read
DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings
conatusai profile
Jahn

DGX Spark (GB10) bare-metal vLLM: the install that works, two landmines, measured timings

#nvidia #llm #gpu #vllm
Comments
2 min read
Ollama to vLLM: When to Migrate Your Local LLM Server
rosgluk profile
Rost

Ollama to vLLM: When to Migrate Your Local LLM Server

#ollama #vllm #llm #ai
Comments
15 min read
The unofficial TPU migration guide: Cloud TPU API to Compute Engine
xbill profile

The unofficial TPU migration guide: Cloud TPU API to Compute Engine

#tpu #gcp #vllm #devops
6
Comments 2
17 min read
Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity
xbill profile

Self-Hosted Gemma 4 on TPU v6e: Deployment & SRE with Antigravity

#tpu #llm #vllm #antigravity
Comments
8 min read
What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage
mingxintech profile

What 90% Line-Rate Utilization on a Single 100GbE Port Means: Analyzing Network Bottlenecks in Inference Storage

#kvcache #lmcache #vllm #ai
Comments
5 min read
Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't
xbill profile

Serving Gemma 4 E2B on a TPU v6e-1: what Trillium buys, and what it doesn't

#tpu #vllm #llm #gcp
2
Comments
20 min read
Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU
xbill profile

Running Gemma 4 on EC2 G5g: Graviton2 AMD with NVIDIA GPU

#aws #vllm #cuda #machinelearning
13
Comments
9 min read
Serving Gemma4 with Rust on vLLM 🦀
xbill profile

Serving Gemma4 with Rust on vLLM 🦀

#rust #vllm #aws #cuda
10
Comments
10 min read
Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1
xbill profile

Self-hosting a lite agent backend on one TPU: Gemma 4 E2B + vLLM on a v5e-1

#tpu #vllm #llm #gcp
16
Comments 1
21 min read
Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)
sikamikanikobg profile

Does a Second GPU Increase Ollama's Context Window? (Quadro P2000 + RTX 3090 Tested)

#llm #ollama #vllm #gpu
Comments
3 min read
vLLM vs llama.cpp vs Ollama: What Happens When Your Model Doesn't Fit in 24GB VRAM
sikamikanikobg profile

vLLM vs llama.cpp vs Ollama: What Happens When Your Model Doesn't Fit in 24GB VRAM

#llm #homelab #vllm #ai
Comments
6 min read
👋 Sign in for the ability to sort posts by relevant, latest, or top.