Skip to content
Lead Data Engineer – Identity Targeting Solutions
| Company | Kargo |
|---|
| Location | New York, NY, USA |
|---|
| Salary | $180000 – $200000 |
|---|
| Type | Full-Time |
|---|
| Degrees | |
|---|
| Experience Level | Senior, Expert or higher |
|---|
Requirements
- Proven expertise in owning, designing and maintaining large-scale, interdependent systems
- Experience in planning deployment and leading a team to complete development on such systems
- Mastery in Python and experience with building complex pipelines in Airflow
- Proficiency in crafting SQL queries for Snowflake, with an understanding of cost and performance nuances
- Experience with using Spark to process large volumes of data and understand the nuances of how to structure data for efficient querying
- Experience with Docker/Kubernetes and the AWS ecosystem
- Familiarity interacting with third party APIs within ETL pipelines
- Familiar with Agile methodologies and have worked with Jira/Confluence
- Comfortable communicating and presenting complex topics and plans to people in the organization, regardless of their technical skill level
- Comfortable with requirements gathering, cross-team design brainstorming sessions, and ensuring consensus among stakeholders
- Innate drive to take initiative and ownership of projects, transform uncertainty into actionable strategies, and a deep-seated pride in delivering meticulous, high-quality work
Responsibilities
- Oversee a team of data engineers, ensuring adherence to high standards, project delivery, and continuous skill development
- Engage in deep-dive sessions with a diverse range of technical and non-technical stakeholders, translating business needs into actionable technical requirements in Jira/Confluence
- Collaborate with data vendors and our data partnership team to evaluate and onboard new datasets that will enhance our targeting abilities
- Build and maintain scalable and cost-efficient data ingestion pipelines (Python, Airflow, Snowflake, AWS, Docker, Kubernetes)
- Spearhead the architecture, design and enhancement of our targeting solutions and workflows, ensuring quality, efficiency, and transparency of the processes
- Create comprehensive documentation detailing system architecture, interdependencies, data flow diagrams, and best practices for future reference and onboarding purposes
- Monitor data storage and processing costs, seeking ways to optimize without compromising on quality
Preferred Qualifications
- Experience with identity and data products within AdTech is a big plus
