AI Platform Engineer (LLM Platform & Operations)

Listed as a remote role based in Sofia, Sofia City, Bulgaria. Own the platform our AI agents run on We are hiring one engineer to own the platform our AI agents run on. That platform is real and running today: agents take a work item, plan it, write code, review it and open a pull request with a human name on the approval. It executes against production codebases. It costs real money every month. What it does not yet have is an owner whose job is to make it observable, affordable, reproducible and safe to change. That is this role. You will not be asked to train models. You will be asked to run the system that calls them, and to make it something the whole engineering organisation can depend on. A greenfield-heavy role inside a working system — the rare combination where the thing already produces value and most of the platform engineering around it is still yours to design. What you will own: The runtime. A self-managed Kubernetes estate on AWS, a durable workflow engine orchestrating multi-stage agent pipelines, and long-lived agent workers. Making it reproducible from code, and rebuildable by more than one person. The telemetry. Token and cost per agent, per task and per engineer; adoption and quality metrics that come off a dashboard rather than out of a spreadsheet. Today this does not exist, and it blocks our ability to prove what the platform is worth. Cost. Model spend is currently observed, not governed. You will introduce per-run and per-team budgets, quotas and anomaly alerting, and own the monthly cost story to the CFO. Evaluation. Build and run the harness that answers “did this change make the agent worse?” — with regression evaluations gating CI, so the answer arrives before a merge rather than during a demo. Retrieval. The code-intelligence layer that lets an agent find a symbol, its callers and its risk inside a large repository within a fixed token budget, instead of reading everything. And the shared, source-traceable knowledge layer that agents retrieve from. Reliability and governance. Service levels, alerting that actually pages, on-call, unified authentication, short-lived credentials in place of standing secrets. What we are looking for: Production Kubernetes — operated, debugged and rebuilt, not just deployed onto. Self-managed clusters are a strong plus. Serving LLMs in production behind a gateway — routing, fallbacks, rate limits, retries, and what a burst of throttling does to a queue of workers. LLM observability and cost control — you have instrumented token and cost telemetry per request, model and user, and acted on what it showed. Evaluating non-deterministic systems — you have built or run an evaluation harness, and can say plainly what it caught and what it missed. Retrieval built for machines, not people — RAG, code indexing, symbol-level navigation, or search consumed by an automated caller. Equally valuable: the judgement to know when retrieval is the wrong tool. Cloud identity and secrets in depth , AWS primarily. OpenTelemetry, Prometheus, Grafana as a working habit rather than a retrofit. Infrastructure as code , and the instinct that a hand-built environment is a defect. Python — enough to work confidently in service and agent codebases and to review other people's code. You have carried a pager for something that mattered, and can describe an incident you made worse before you made it better. Nice to have: Durable workflow engines (Temporal, or similar). Agent runtimes such as LangGraph. Policy engines — OPA, OpenFGA, Cedar. Dynamic secrets management. Model Context Protocol servers. PostgreSQL with pgvector alongside transactional load. Azure DevOps Pipelines. Sandbox isolation — gVisor, Kata, Firecracker. Exposure to a second cloud. If you have run LLMOps under that name somewhere, we would like to hear about it — the title is not yet standard, and the work is what matters. What we are not looking for: No model training, fine-tuning or RLHF. No research publications. No data science background. No specific degree. Prompt writing is not the job. Show more Show less Seniority level Mid-Senior level Employment type Full-time Job function Engineering Industries IT Services and IT Consulting and Financial Services