DE
Full time
🤖 AI / ML
DevOps Engineer - AI Runtime Services at Demandbase
Listed by DevShelfHub
India (offices also in San Francisco, New York, Austin, Seattle, UK)
Hybrid
Mid-level · 2–5 yrs
About the Role
Demandbase is hiring a DevOps Engineer for its AI Runtime Services team to build and operate the shared runtime powering LLM features across the platform. As enterprises scale account-based GTM with AI, infrastructure roles that combine SRE ownership with LLM gateway, observability, and cost control expertise are in high demand.
This is a genuine production ownership role — you'll operate the LLM gateway, reliability systems, and internal AI tooling that product teams depend on daily.
What you'll do
- AI Runtime Services owns the shared runtime that AI at Demandbase runs on — enabling product teams to ship LLM features and supporting internal AI solutions.
- Own the LLM gateway as the single front door to every model provider
- Manage reliability with SLOs, on-call rotations, incident response, and postmortems
- Implement cost control with per-team attribution, budgets, and quotas
- Build LLM observability for traces, spans, and prompt/response capture
- Develop evals and experimentation infrastructure for model and prompt changes
- Deploy caching and guardrails for latency, spend reduction, and safety
- Move across gateway, ingest pipelines, and internal tooling as team needs evolve
- Operate production systems on Kubernetes, AWS, Terraform, and GitOps
What we're looking for
- Strong production infrastructure background: Kubernetes, AWS, Terraform, GitOps
- Python at a level where you can own services and tooling
- Real on-call and incident-response experience
- Observability fluency — instrumented systems, metrics/logging backends, meaningful alerts
Skills & Technologies
Required
Strong production infrastructure background: Kubernetes, AWS, Terraform, GitOps
Python at a level where you can own services and tooling
Real on-call and incident-response experience
Observability fluency — instrumented systems, metrics/logging backends, meaningful alerts
Hands-on LLM systems experience — built agents, RAG pipelines, or internal tools with evals
Familiarity with tokens, context windows, prompt/response tracing, and eval suites
Preferred Skills
LLM gateway or proxy (LiteLLM or similar) in production
Benefits & Perks
- Health Insurance
- Flexible Leave Policy
- Learning Budget
- EPF / NPS
Why This Role is Good for Experienced Professionals
- This role offers end-to-end ownership of AI runtime infrastructure at a fast-growing B2B AI platform.
- Own the LLM gateway — keys, routing, rate limits, failover, and provider quotas
- Genuine SRE ownership with SLOs, on-call, incident response, and postmortems
- Build cost control systems with per-team and per-model attribution and budgets
- Implement LLM observability — traces, spans, prompt/response capture, and billing visibility
- Develop evals and experimentation infrastructure for product teams
- Deploy caching and guardrails for safety, PII handling, and policy enforcement
- Work across gateway, ingest pipelines, product runtime, and internal tooling
- Operate on a modern stack: EKS, Flux/GitOps, Karpenter, LiteLLM, Datadog
- Join an early team with broad scope and high impact across all AI features
- Contribute to a company recognized for culture and career growth
About Demandbase
Demandbase is a pipeline AI platform that empowers GTM teams to automate growth at scale. With a unified view of data, insights, actions, and outcomes, B2B enterprises align and execute account-based GTM strategies on one platform. Thousands of businesses trust Demandbase to maximize revenue, minimize waste, and consolidate data and tech stacks.
The company invests heavily in people and culture, recognized as one of the Best Places To Work in the San Francisco Bay Area by Fortune and one of the 60 Best Companies To Sell For by Selling Power.
The company invests heavily in people and culture, recognized as one of the Best Places To Work in the San Francisco Bay Area by Fortune and one of the 60 Best Companies To Sell For by Selling Power.
- Industry
- Technology
- Company Size
- 500+
- Website
- demandbase.com
🚀 Boost your chances
Skip the queue
Contact the recruiter directly at Demandbase and follow up personally — most applicants never do.
Browse Recruiter DatabaseFree trial · 40 contacts · ₹0
Job Details
- Type
- Full time
- Level
- Mid-level
- Experience
- 2–5 years
- Location
- India (offices also in San Francisco, New York, Austin, Seattle, UK)
- Work Mode
- Hybrid
- Category
- AI / ML
Share this Job
Recruiter Database
Want to stand out? Talk to the recruiter directly.
Most applicants never hear back. Our Recruiter Database gives you direct email access to the people making hiring decisions — so you can follow up personally and actually get noticed.
- Recruiter name, company & direct email address
- Sourced from active job postings across top companies
- Free trial — 40 contacts, no credit card needed