Senior IT App Engineer

At Klaviyo, we value the unique backgrounds, experiences and perspectives each Klaviyo (we call ourselves Klaviyos) brings to our workplace each and every day. We believe everyone deserves a fair shot at success and appreciate the experiences each person brings beyond the traditional job requirements. If you’re a close but not exact match with the description, we hope you’ll still consider applying. Want to learn more about life at Klaviyo? Visit klaviyo.com/careers to see how we empower creators to own their own destiny.

At Klaviyo, the ARIA (Applications, Resources, Intelligence, and Automations) team is building the AI systems that transform how our employees work. We are looking for a Senior AI Engineer to design, build, and scale production-grade LLM integrations and AI-powered automation tools from the ground up. This role sits at the intersection of applied AI research and full-stack software engineering. You will own everything from LLM integration architecture and RAG system design to CI/CD pipeline management and production observability. If you want to build AI that real people use at scale every day, this is the role for you.

You will report into the Director of AI and App Engineering and ….

How you’ll make a difference:

• Design and build LLM-powered applications: Architect and develop production-grade systems leveraging large language models (GPT-4, Claude, Mistral, Llama) for internal automation, knowledge retrieval, and intelligent workflow assistance.
• Build and maintain AI integration pipelines: Design robust, scalable API integrations between AI models and internal systems such as HRIS, ticketing, CRM, and knowledge bases, ensuring reliability, low latency, and high availability.
• Develop full-stack AI features end-to-end: Own the complete development of AI features across the stack from backend model orchestration and API layers to frontend interfaces that make AI accessible to non-technical employees.
• Architect and manage CI/CD pipelines for AI systems: Build and maintain automated deployment pipelines for AI models and services, including model evaluation frameworks, A/B testing infrastructure, and safe rollout strategies.
• Implement RAG and retrieval systems: Design and build retrieval-augmented generation systems grounded in Klaviyo internal knowledge, ensuring accuracy, relevance, and up-to-date responses at scale.
• Evaluate and fine-tune AI models: Lead benchmarking, evaluation, and fine-tuning of foundation models for Klaviyo use cases, including prompt engineering and parameter-efficient fine-tuning techniques.
• Establish AI engineering best practices: Define coding standards, testing frameworks, and architectural patterns for AI development at Klaviyo, enabling the team to ship reliable AI features consistently.
• Build AI observability and monitoring tooling: Instrument AI systems with latency monitoring, output quality evaluation, hallucination detection, cost dashboards, and production alerting.
• Collaborate with Product and Data Science: Work closely with PMs, data scientists, and ML engineers to translate requirements into technical implementations, maintaining a tight feedback loop between experimentation and production.
• Drive security and compliance in AI systems: Ensure AI integrations follow data privacy best practices, implement guardrails for sensitive data handling, and maintain compliance with Klaviyo security and governance standards.

Who you are:

• 3+ years of software engineering experience, with at least 2 years focused on building AI, ML, or LLM-powered systems in production.
• Deep expertise with large language models such as OpenAI, Anthropic, Mistral, or Llama, and experience integrating them into production applications via API.
• Strong full-stack engineering skills across Python (FastAPI, Flask, or Django) and TypeScript or JavaScript, with experience in modern frontend frameworks.
• Proven experience designing and building CI/CD pipelines for software or ML systems using tools such as GitHub Actions, Jenkins, or CircleCI.
• Experience with RAG systems, vector databases (Pinecone, Weaviate, pgvector), and embedding models.
• Proficiency with cloud infrastructure (AWS, GCP, or Azure), containerization (Docker, Kubernetes), and infrastructure-as-code tools such as Terraform or Pulumi.
• Experience with prompt engineering, model evaluation frameworks, and LLM observability tools such as LangSmith or Weights and Biases.

We use Covey as part of our hiring and / or promotional process. For jobs or candidates in NYC, certain features may qualify it as an AEDT. As part of the evaluation process we provide Covey with job requirements and candidate submitted applications. We began using Covey Scout for Inbound on April 3, 2025.

Please see the independent bias audit report covering our use of Covey here

Massachusetts Applicants:
It is unlawful in Massachusetts to require or administer a lie detector test as a condition of employment or continued employment. An employer who violates this law shall be subject to criminal penalties and civil liability.

Our salary range reflects the cost of labor across various U.S. geographic markets. The range displayed below reflects the minimum and maximum target salaries for the position across all our US locations. The base salary offered for this position is determined by several factors, including the applicant’s job-related skills, relevant experience, education or training, and work location.

In addition to base salary, our total compensation package may include participation in the company’s annual cash bonus plan, variable compensation (OTE) for sales and customer success roles, equity, sign-on payments, and a comprehensive range of health, welfare, and wellbeing benefits based on eligibility.

Your recruiter can provide more details about the specific salary/OTE range for your preferred location during the hiring process.

This role may require up to 10% travel for purposes such as new hire onboarding, client or partner work if applicable, team meetings, and industry events. Travel is coordinated in advance.

Get to Know Klaviyo

We’re Klaviyo (pronounced clay-vee-oh). We empower creators to own their destiny by making first-party data accessible and actionable like never before. We see limitless potential for the technology we’re developing to nurture personalized experiences in ecommerce and beyond. To reach our goals, we need our own crew of remarkable creators—ambitious and collaborative teammates who stay focused on our north star: delighting our customers. If you’re ready to do the best work of your career, where you’ll be welcomed as your whole self from day one and supported with generous benefits, we hope you’ll join us.

AI fluency at Klaviyo includes responsible use of AI (including privacy, security, bias awareness, and human-in-the-loop). We provide accommodations as needed.

By participating in Klaviyo’s interview process, you acknowledge that you have read, understood, and will adhere to our Guidelines for using AI in the Klaviyo interview Process. For more information about how we process your personal data, see our Job Applicant Privacy Notice.

Klaviyo is committed to a policy of equal opportunity and non-discrimination. We do not discriminate on the basis of race, ethnicity, citizenship, national origin, color, religion or religious creed, age, sex (including pregnancy), gender identity, sexual orientation, physical or mental disability, veteran or active military status, marital status, criminal record, genetics, retaliation, sexual harassment or any other characteristic protected by applicable law.