Prompt Engineer / LLM Engineer (Generative AI) – Python (LATAM)
Posted on 21 October 26 by Laks
- $ - $
Powered by Tracker
Job Description
Prompt Engineer / LLM Engineer – Python & Text-to-QueryLocation: Latin America (LATAM) – Fully Remote
Argentina, Uruguay, Chile, Costa Rica, Brazil, Colombia and Ecuador
Employment Type: Contract
Experience: 2+ years of Python experience
Key Responsibilities
Design, restructure, maintain, and version prompts, including domain taxonomies, query grammar, examples, and guardrails.
Build and maintain an evaluation harness using real-world user questions to measure accuracy, latency, and cost.
Expand support for different question types, ambiguous phrasing, domain-specific synonyms, and newly searchable fields.
Develop Python components for prompt assembly, structured intent parsing, validation, query generation, and automated evaluation.
Improve explainability by making generated queries, uncertainty, and refusal scenarios understandable to end users.
Evaluate LLMs from multiple providers and compare alternative approaches, including decision-style models and smaller fine-tuned models.
Validate generated queries to ensure they follow defined rules and execute safely.
Establish regression testing and evidence-based evaluation for prompt, model, and system changes.
Collaborate with the client's technical lead to deliver improvements aligned with technical and business requirements.
Required Skills and Qualifications
Hands-on prompt engineering experience in a shipped or production product, beyond experimentation or proof-of-concept development.
Strong Python development skills, with at least 2 years of experience.
Experience building natural-language-to-query solutions, such as Text-to-SQL, Solr, Elasticsearch, or similar systems.
Experience with structured LLM outputs, few-shot prompting, retrieval-augmented prompting, and domain vocabulary handling.
Experience integrating LLM APIs from at least two providers.
Understanding of query validation, safe execution, error handling, and output validation.
Experience designing evaluation frameworks to measure accuracy, latency, cost, and regression performance.
Understanding of LLM context limits, model drift, and production performance trade-offs.
Ability to design and explain architectures that prevent LLMs from accessing protected or row-level data.
Preferred Qualifications
Experience with Apache Solr query construction, search, and faceting.
Familiarity with healthcare or EMS terminology, such as NEMSIS, ICD, or RxNorm.
Experience fine-tuning small language models or evaluating alternative model architectures.
Familiarity with LLM evaluation frameworks.
Exposure to Azure AI, Microsoft Fabric, or related Microsoft AI technologies.