Software Engineer - ML Infrastructure

Software Engineer – ML Infrastructure

BS or MS in computer science (or equivalent work experience) with focus in data engineering and machine learning
2+ years of industry experience developing big data processing and/or machine learning pipelines
1+ years of hands on experience with cloud platforms (AWS/GCP/Azure)
Proficient in Python
Solid understanding of system design fundamentals and distributed computing concepts

Develop and maintain highly scalable data processing pipelines for data curation, annotation, search and ml feature extraction.
Build and improve our active learning pipelines. Ensure the scalability of our training jobs and inference endpoints.
Develop and maintain our orchestration and scheduling systems.
Collaborate with autonomy engineers to build new features for our autonomy data platform, improving data accessibility and developer productivity.
Build integrations with annotation providers to efficiently annotate large scale datasets
Develop infrastructure components using IaC(Infrastructure as Code) and implement CI/CD processes to streamline ongoing development of the platform.
Develop monitoring and alerting frameworks to ensure platform reliability, stability and cost efficiency.
Collaborate with ML Engineers in accelerating the ml development velocity.
Work with the data team to ensure SLAs around data quality and availability. Optimise storage costs by tuning retention policies and improving data access patterns.