job summary: We are looking for an experienced software engineer who can build tools and solutions for AI evaluations, content aggregation, business rule improvements, prompt tuning, and model changes, and serve as a technical liaison with our operational product partners to optimize the platform over time and evaluate and mitigate poor/unexpected output. You will be the go-to technical partner bridging product operations and engineering for our search and answer platform. You'll investigate and resolve specific question issues, identify patterns across classes of issues, and drive pragmatic solutions-from code fixes and prompt updates to content and business rule changes. You'll run experiments, track metrics, and coordinate changes with multiple teams, ensuring quality and reliability in a hypercare environment, especially around releases. location: Orlando, Florida job type: Contract salary: $84 - 88 per hour work hours: 9am to 5pm education: No Degree Required responsibilities: - Work with product partners to design and run experiments for changes to models, prompts, content, retrieval strategies, guardrails, and other variables; track and report metrics using internal tooling and industry-standard AI observability platforms. - Build or enhance tooling for content gap detection, retrieval diagnostics, prompt evaluation, and automated guardrails to minimize poor/unexpected outputs. - Help define and monitor key KPIs (e.g., precision/recall, MRR/NDCG, answer accuracy, coverage, and latency) and maintain dashboards that reflect quality and reliability over time. - Investigate and determine root cause for specific search and answer issues in partnership with product and operations (e.g., content gaps, retrieval/pipeline problems, prompt quality, code bugs, configuration or business rule errors). - Implement fixes directly or coordinate with the appropriate engineering and content teams to mitigate issues and prevent regressions. - Identify classes of issues, propose product improvements, and design technical solutions to improve relevance, answer quality, and recommendation performance at scale. - Own triage in a hypercare role after releases: ensure issues are logged, prioritized, assigned, and driven to closure with clear communication and SLAs. - Document findings, playbooks, and runbooks; create repeatable processes for RCA, experimentation, and hypercare operations. qualifications: - Strong software engineering skills in Java and Python, with experience building back-end services, APIs, pipelines, and integrations. - Hands-on experience with search technologies and Retrieval-Augmented Generation (RAG), including vector embeddings, chunking strategies, and prompt building. - Practical knowledge of when and how to use LLMs to improve efficiency and quality, including prompt design/tuning, guardrails, and fallback strategies. - Proven root cause analysis (RCA) capabilities: ability to trace issues across data, retrieval, prompts, models, and code; skilled at identifying patterns and proposing systemic fixes. - Excellent communication, relationship-building, and cross-team coordination skills; comfortable serving as a liaison between product operations and engineering. - Experience with experimentation frameworks and A/B testing, defining success metrics, and interpreting results to drive decisions. - Familiarity with AI observability and monitoring tools for prompt/model performance, drift detection, and error tracking. - Proficiency with CI/CD, version control, automated testing, and feature flagging. - Strong bias for action; able to manage multiple priorities in a fast-moving environment and follow through from issue detection to mitigation. Equal Opportunity Employer: Race, Color, Religion, Sex, Sexual Orientation, Gender Identity, National Origin, Age, Genetic Information, Disability, Protected Veteran Status, or any other legally protected group status.At Randstad Digital, we welcome people of all abilities and want to ensure that our hiring and interview process meets the needs of all applicants. If you require a reasonable accommodation to make your application or interview experience a great one, please contact HRsupport@randstadusa.com. Pay offered to a successful candidate will be based on several factors including the candidate's education, work experience, work location, specific job duties, certifications, etc. In addition, Randstad Digital offers a comprehensive benefits package, including: medical, prescription, dental, vision, AD&D, and life insurance offerings, short-term disability, and a 401K plan (all benefits are based on eligibility). This posting is open for thirty (30) days.