Skip to content
Conversational AI Specialist – Director
| Company | Morgan Stanley |
|---|
| Location | New York, NY, USA |
|---|
| Salary | $90000 – $155000 |
|---|
| Type | Full-Time |
|---|
| Degrees | |
|---|
| Experience Level | Senior, Expert or higher |
|---|
Requirements
- Experience with Conversational AI solutioning / product development / experimentation or Innovation.
- Experience with prompt engineering for using Commercial or Open Source LLMs such as GPT, Claude, Gemini, LLAMA.
- Experience with conversational AI evaluation techniques such as F1, BLEU, ROUGE, and human feedback analysis.
- Strong understanding of artificial intelligence concepts and technologies.
- Excellent communication, interpersonal and leadership skills.
- Proven experience in program or project management.
- Ability to manage multiple projects simultaneously in a fast-paced environment.
- Solid analytical and problem-solving abilities with a keen attention to detail.
Responsibilities
- Develop and refine context-aware prompts tailored to different use cases, domains, and user intents with a view of improving the accuracy, relevance and efficiency of AI generated chatbot responses.
- Develop and maintain robust frameworks for testing and evaluating Generative AI responses for coherence, bias, factual accuracy and user satisfaction.
- Identify and troubleshoot issues and anomalies in the LLM model outputs and provide feedback and recommendations for solution improvement and optimization.
- Establish and maintain effective communication channels with internal stakeholders such as tech partners and business product owners.
- Collaborate with cross-functional teams, including internal stakeholders and vendor partners.
- Identify potential risk and issues that may impact project delivery and develop mitigation strategies to address them proactively.
- Track project performance metrics and KPIs to measure progress against objective and identify areas for improvement.
Preferred Qualifications
No preferred qualifications provided.
