Senior Snowflake Data Engineer – AVP
| Company | State Street |
|---|---|
| Location | Toronto, ON, Canada |
| Salary | $Not Provided – $Not Provided |
| Type | Full-Time |
| Degrees | |
| Experience Level | Senior, Expert or higher |
Requirements
- More than 7+ years of experience in IT
- Strong experience in design/data modeling/development & performance tuning
- Experience in building Cloud based DW solutions
- Experience with Azure, Snowflake, DBT, Airflow, Cosmos DB
- Ability to optimize and tune Snowflake performance
- Experience in troubleshooting DevOps pipelines and Azure services
- Experience in handling semi-structured data (JSON, XML) in Snowflake
Responsibilities
- Design, build and test end to end data pipeline including data ingestion (streaming, events, and batch), data integration, data curation
- Design, develop, and deploy scalable data pipelines and ETL processes on cloud-based infrastructure using Azure, Snowflake, DBT, Airflow, Cosmos DB
- Define and implement automation of jobs and testing
- Optimize the data pipeline to support workloads and use cases
- Support mission critical applications and near real time data needs from the data platform
- Address data issues, environment issues, performing impact analysis, root cause analysis, corrective, adaptive and perfective maintenance
- Implement data models, transformations, and schema designs to support analytical and reporting needs
- Optimize and tune Snowflake performance, including query optimization, resource management, and scaling strategies
Preferred Qualifications
- Experience in micro-services architecture and understanding of Cloud Computing
- Experience with MicroServices – API/Event Driven Architecture/Development
- Experience with DBT (Data Build Tool) for data modelling and transformation and Apache Airflow for workflow orchestration and scheduling
- Experience in Snowflake utilities including SnowSQL, Snowpipe, Snowlight for handling Streaming data
- Expertise in Snowflake advanced concepts like setting up resource monitors, RBAC controls, virtual warehouse sizing, query performance tuning, Zero copy clone, time travel and understand how to use these features
- Experience in in re-clustering Snowflake data with good understanding of Micro-Partition within Snowflake
- Expertise in deploying Snowflake features such as data sharing, events and lake-house patterns
- Any development experience on Databricks or Scala is a plus
- Exposure to working in financial services; front- or middle-office, or fund management business areas ideal
- Experience dealing with users or clients from either a technical or business area
- Desire to work in client facing environment
- Experience of working in the financial industry
- Experience with agile development methodology