Senior Prompt Engineer: LLM Migration & Optimization

United States

Mandatory
English
Be one of the first to apply
DESCRIPTION
Are you an expert at navigating the complex architecture of Large Language Models? Welo Data is seeking a Senior Prompt Engineer to lead the technical migration of template workflows into high-performance LLM autoraters.

This role is designed for a technical specialist who understands that "perfecting a prompt" is a rigorous engineering discipline. You will leverage advanced APG/APO tools and manual refinement to ensure our automated systems meet—and exceed—human crowd baselines in accuracy and nuance.

The Mission: Automated Quality at Scale

  • Architectural Migration: Take full ownership of the end-to-end technical migration of templates to LLM autoraters.
  • Optimization Leadership: Utilize Automatic Prompt Generation (APG) and supervise Automated Prompt Optimization (APO) tools to push model performance past plateaus and deadlocks.
  • Metrics-Driven Excellence: Continuously measure quality against "gold data" baselines, tracking precision, recall, and $F_1$ scores to justify launch readiness.
  • Edge-Case Engineering: Manually draft and refine complex prompts to overcome anti-patterns and architecture gaps that automated tools can't solve.

Project Details
  • Job Title: Senior Prompt Engineer (LLM Migration)
  • Schedule: Part-Time (Set your own hours within project milestones)
  • Location: 100% Remote (Must be based in the United States)
  • Employment Type: Freelance / Independent Contractor

Candidate Profile

  • Educational Foundation: Bachelor’s, Master’s, or PhD in Computer Science, Data Science, Computational Linguistics, or a related analytical field.
  • Prompt Engineering Mastery: 4+ years of experience tuning LLMs for strict, structured outputs, complex classification, and few-shot learning.
  • Analytical Power: High proficiency in identifying error patterns and using SQL or data analytics tools to monitor performance.
  • Technical Agility: Fast learner capable of mastering proprietary internal tools and "Goose API" style interfaces with minimal oversight.

Preferred Technical Skills

  • Familiarity with shadowbot monitoring and disagreement tracking.
  • Experience in AI model evaluation and software engineering.
  • Deep understanding of semantics, logic, and Chain-of-Thought (CoT) prompting.
  • Proven ability to draft high-level Launch Certification Documentation.

Recruitment & Onboarding
  • Technical Review: Submit your CV and portfolio of LLM optimization work.
  • Prompt Assessment: Demonstrate your ability to navigate complex template clusters and APO deadlocks.
  • Tooling Deep-Dive: Get access to our internal technical suite and migration workflows.
  • Launch: Begin your part-time engagement and lead the shift to LLM-driven autorating.
SKILLS
Prompt
Prompt Delivery
Large Language Models
It Skills
Automatic Prompt Generation

ARE YOU THE PERFECT CANDIDATE?
Be one of the first to apply