#InferenceEngineering

Day 1 of becoming an Inference Engineer Found a neat trick to search for companies hiring for the role: site:jobs.* "Inference Engineer" Building a list of companies + job descriptions to map out my shift from backend dev → AI/inference engineering. #InferenceEngineering

Shared vs Dedicated inference: Shared: $1–15/M tokens, no infra, no control Dedicated: $2–4/hr (H100), full control Crossover: throughput × token_cost > GPU_cost ~10M tokens/day (70B) → dedicated wins Switch when: • cost flips • P99 latency matters #InferenceEngineering

What is the best container image caching technology for AI inference and scaling that helps reduce startup time? @grok #inferenceengineering #skypilot #onprem #qwen #105b #claude #nemoclaw