Interim Data Engineer
Posted
About the role:
You will play an important role in the Data infrastructure team, which is responsible for building and maintaining a scalable, robust, self-service analytics platform.
What you'll do:
- Maintaining and improving the ingestion pipeline to reliably deliver billions of events daily in a defined SLA
- Provide support for all teams in building and optimizing their complex pipelines
- Work closely with other teams to identify pain points and problems around the platform
- Develop new tools and frameworks to improve the data platform
- Work in close collaboration with data scientists and data analysts to help support their work in production
- Work with vast projects such as building an ML platform and streaming use cases
- Set up best practices and processes around software and data development
What you bring in:
- 3+ years of experience in data engineering
- Experience working with an orchestration tool (ex, Airflow)
- Experience working with a large data set
- Experience building complex ETL pipelines
- Experience working with cloud providers such as GCP, AWS, or Azure
- Strong programming skills in Spark with Scala and Python
- Experience working with CI/CD tools like Jenkins and Git
- Strong understanding of Software Engineering practices and principles
- Excellent problem-solving and communication skills
- Self-motivated and have a proven ability to take the initiative to own the problems that come up and solve them
Must-have experience:
- Hands-on experience building, scaling, and operating production-grade data platforms or data infrastructure , including reliability, observability, performance, and self-service capabilities.
It would be great if you also have:
- Working with messaging systems like Kafka
- Knowledge of Kubernetes
- Hands-on experience with any streaming platform
- Experience managing a data warehouse in BigQuery or Redshift
This assignment is managed by FILL that has the exclusive end client relationship. Consultants presented to FILL through Commended will be priced at their indicated rate +4%.
Desired skills
Data analysisData scienceBig DataCI/CD (CICD)PythonJenkinsGitSoftware developmentETL (Extract transform and load)Apache KafkaKubernetesInfrastructure, monitoring & performanceAWS (Amazon Web Services)Microsoft AzureGCP (Google Cloud Platform)
Questions about this assignment?
Consultant hiring processes are often quick. We recommend that you apply and share your questions in the note in your application to make sure you don't miss out.