Model Governance Senior Analyst

Model Governance Senior Analyst

Requirements

• Quantitative bachelor’s degree (i.e., computer science, physics, mathematics or engineering)
• 3+ years of hands-on experience in programming and/or ML/AI implementation, including K-means clustering, regressions, large language models (LLM), decision trees, and random forest
• Basic knowledge of prompt engineering for LLM model development and validation
• Intermediate level of programming: Python, No-SQL, XML, and SQL (Oracle)
• Intermediate level of understanding in ML/AI concepts, which includes model algorithms, feature importance, precision, recall, F-measure, and sensitivity analysis
• Ability to debug codes in Python, SQL and Java to perform peer review of code files
• Basic knowledge of Java and Python packages (Scikit-learn, Plotly, Catboost, Seaborn)
• Basic use of Linux, Confluence, Jira and code central repository control (e.g., Bitbucket and Github)
• Ability to work independently and as a team contributor to effectively prioritize tasks in a fast-paced environment
• Ability to communicate technical context to non-technical audience

Responsibilities

• Develop independent codes and programs to conduct above-the-line (ATL) and below-the-line (BTL) tuning on AML and Trade Surveillance models.
• Design efficient programming logic to conduct performance testing on existing machine learning (ML) and artificial intelligence (AI) models focused on model precision, recall, F-measure, and sensitivity.
• Validate algorithm and programming logic in Python, SQL, or Java.
• Develop codes and scripts in Python and SQL to perform quantitative risk based analysis on financial crime data and data quality testing under the guidance of manager.
• Build software prototypes for detecting money laundering and trade manipulation risks by using python and SQL.
• Conduct design analysis on transaction and trade manipulation monitoring systems.
• Perform user acceptance testing on software prototypes to assess performance and feasibility of the software implementation.
• When required, conduct independent development of AI/ML and other models as part of validation or internal research.
• Create and maintain documentation on tuning reports, tuning schedules, and data analyses.
• Review ML/AI model documentation against governance procedures and regulatory expectations as part of testing and tuning.
• Develop and update model tuning plans.
• Upkeep the model governance framework and all supporting documentation.
• Participate in meetings with key model governance and other regional leadership stakeholders to ensure the effective governance and oversight of model identification, development, use, and evaluation processes.
• Assess model documentation, including development, monitoring, and implementation documentation, for completeness against program standards and templates.

Preferred Qualifications

• Regulatory compliance experience working for an exchange, a regulatory organization, a Broker-Dealer (BD), a Futures Commission Merchant (FCM), a Bank or a similar organization