Senior Data Engineer
| Company | Plume |
|---|---|
| Location | Palo Alto, CA, USA |
| Salary | $127000 – $149000 |
| Type | Full-Time |
| Degrees | Bachelor’s, Master’s, PhD |
| Experience Level | Senior |
Requirements
- BS/MS/PhD in Computer Science, Electrical Engineering or related technical field
- 6+ years of software development experience with a proven track record of building, scaling, and supporting production data pipelines
- Intermediate proficiency in writing clean, idiomatic code in Scala or Python
- Intermediate proficiency in writing SQL in data warehousing technologies
- Good understanding of distributed data processing technologies (e.g. Apache Spark) and handling data at petabyte scale
- Good understanding of data warehousing concepts
- Good analytical and problem-solving skills
- Good oral and written communication skills
Responsibilities
- Architect and productionize Spark data pipelines supporting data-oriented products, machine learning, and analytics
- Work with cross-functional teams to design end-to-end system architectures and data flows
- Adhere to data protection requirements including data access, retention, residency, and de-identification
- Maintain up-to-date documentation of data warehouse schemas
- Systematically enhance data quality, availability, and observability
- Analyze and solve performance, cost, and scale challenges
- Refactor code as needed to improve performance and simplify operations
- Provide production support in triaging and fixing issues relating to data quality and availability
- Mentor junior team members and new hires
Preferred Qualifications
-
No preferred qualifications provided.