كلما زادت طلبات التقديم التي ترسلينها، زادت فرصك في الحصول على وظيفة!

إليك لمحة عن معدل نشاط الباحثات عن عمل خلال الشهر الماضي:

عدد الفرص التي تم تصفحها

عدد الطلبات التي تم تقديمها

استمري في التصفح والتقديم لزيادة فرصك في الحصول على وظيفة!

هل تبحثين عن جهات توظيف لها سجل مثبت في دعم وتمكين النساء؟

اضغطي هنا لاكتشاف الفرص المتاحة الآن!
نُقدّر رأيكِ

ندعوكِ للمشاركة في استطلاع مصمّم لمساعدة الباحثين على فهم أفضل الطرق لربط الباحثات عن عمل بالوظائف التي يبحثن عنها.

هل ترغبين في المشاركة؟

في حال تم اختياركِ، سنتواصل معكِ عبر البريد الإلكتروني لتزويدكِ بالتفاصيل والتعليمات الخاصة بالمشاركة.

ستحصلين على مبلغ 7 دولارات مقابل إجابتك على الاستطلاع.


تم إلغاء حظر المستخدم بنجاح
شكراً لك، تم إرسال بلاغك وسيتم مراجعته قريباً.
https://bayt.page.link/98KrDsXVW8PEry6W9
العودة إلى نتائج البحث‎
Management Solutions International logo

Senior LLM Evaluation Engineer / Applied AI Engineer

Management Solutions International
قبل يومين 2027/03/16 ·ينتهي التقديم خلال 118 يومًا
الدوحة، قطر
خدمات الدعم التجاري الأخرى
أنشئ تنبيهًا وظيفيًا لوظائف مشابهة

الوصف الوظيفي

Senior LLM Evaluation Engineer / Applied AI Engineer

Experience: 7–12 Years
Education: Master's Degree or Higher



Location : Doha



About the Role

We are looking for an experienced Senior LLM Evaluation Engineer / Applied AI Engineer to develop and implement evaluation frameworks for AI-powered applications, Large Language Models (LLMs) and autonomous AI agents.



The role focuses on measuring, testing and improving the quality, accuracy, reliability and performance of AI systems in production.



The ideal candidate will combine strong Machine Learning and AI engineering expertise with advanced Python programming, statistical analysis and hands-on experience in LLM evaluation.



You will be responsible for developing automated evaluation pipelines, designing benchmarking frameworks, implementing LLM-as-a-Judge methodologies and conducting statistical experiments to assess and improve AI system performance.



This is an opportunity to work on advanced Generative AI and Agentic AI systems, ensuring that AI applications deliver accurate, relevant, reliable and consistent results.



Key Responsibilities
  • Design, develop and maintain evaluation frameworks for Large Language Models (LLMs) and Generative AI applications.



  • Build automated evaluation pipelines and testing systems using Python.



  • Develop evaluation datasets, benchmarks, ground-truth datasets and annotation frameworks.



  • Implement LLM-as-a-Judge methodologies to assess AI-generated responses.



  • Evaluate AI systems for factual accuracy, relevance, hallucinations, consistency and response quality.



  • Conduct A/B testing and comparative analysis of different models, prompts and AI configurations.



  • Apply statistical methods, hypothesis testing and confidence intervals to validate model improvements.



  • Evaluate AI agents, tool calling, function calling and multi-step agent workflows.



  • Develop offline and online evaluation methodologies for production AI systems.



  • Monitor model performance and identify opportunities to improve AI system quality, reliability and efficiency.



  • Collaborate with ML Engineers, Data Scientists and AI Researchers to improve model performance and evaluation processes.



Required Technical Skills

Programming & Software Engineering



  • Advanced Python programming



  • Object-Oriented Programming



  • Python-based automation and evaluation pipelines



  • NumPy, Pandas and SciPy



Machine Learning & Generative AI



  • Machine Learning and Deep Learning



  • Large Language Models (LLMs)



  • Generative AI



  • GPT / OpenAI APIs



  • RAG (Retrieval-Augmented Generation)



  • PyTorch / TensorFlow / Scikit-learn



LLM Evaluation & Benchmarking



  • LLM Evaluation



  • LLM-as-a-Judge



  • Model Evaluation and Benchmarking



  • Automated and Human Evaluation



  • Offline and Online Evaluation



  • Hallucination Detection



  • RAG Evaluation



  • Evaluation Frameworks such as RAGAS, DeepEval, TruLens and LangSmith



Statistical Analysis & Experimentation



  • A/B Testing



  • Hypothesis Testing



  • Statistical Significance



  • Confidence Intervals



  • Experimental Design



  • Statistical Analysis



  • Inter-Rater Agreement



AI Agents & Prompt Engineering



  • Agentic AI



  • AI Agent Evaluation



  • Tool Calling and Function Calling



  • Prompt Engineering and Optimisation



  • LangChain / LlamaIndex / Semantic Kernel



Cloud & Data Technologies



  • Microsoft Azure



  • Azure OpenAI



  • Azure AI Services



  • PostgreSQL



  • Elasticsearch



  • SQL



Experience & Qualifications
  • 7–12 years of relevant experience in Machine Learning Engineering, Applied AI, AI Research or Generative AI Engineering.



  • Master's degree or higher in Computer Science, Artificial Intelligence, Machine Learning, Data Science, Statistics, Applied Mathematics or a related discipline.



  • Strong hands-on Python programming and software engineering capabilities.



  • Proven experience developing and evaluating ML/LLM-based applications.



  • Practical experience designing evaluation datasets, benchmarking systems and automated evaluation pipelines.



  • Strong understanding of statistical experimentation, A/B testing and hypothesis testing.



  • Experience with LLM evaluation methodologies, including LLM-as-a-Judge and human evaluation.



  • Exposure to production AI systems, RAG applications and AI agent architectures.



  • Experience with Microsoft Azure and cloud-based AI services is preferred.



Ideal Candidate Profile

The ideal candidate will have experience as a Senior Machine Learning Engineer, LLM Engineer, Applied Scientist, AI Research Engineer or Applied AI Engineer, with a strong specialisation in evaluating and improving AI/LLM systems.



We are particularly interested in professionals who can combine hands-on AI engineering with scientific evaluation methodologies to identify model weaknesses, detect hallucinations, benchmark performance and establish measurable improvements in AI system reliability.



Experience with autonomous AI agents, LLM observability, evaluation frameworks and production-grade Generative AI applications will be highly advantageous.




لقد تمت ترجمة هذا الإعلان الوظيفي بواسطة الذكاء الاصطناعي وقد يحتوي على بعض الاختلافات أو الأخطاء البسيطة.
هل أنت متأكد أنك تريد سحب طلب التقديم إلى هذه الوظيفة؟

لن يتم النظر في طلبك لهذة الوظيفة، وسيتم إزالته من البريد الوارد الخاص بصاحب العمل.