Senior Data Engineer Python Scala ETL
Recruiter
Listed on
Location
Salary/Rate
Type
This job has now expired please search on the home page to find live IT Jobs.
Senior Data Engineer (Python Scala ETL Luigi Airflow Oozie Spark). Would you like to work with complex Big Data technologies within a supportive Agile team environment with flexible working opportunities, casual dress code and early finish on Fridays?
Global software house that produce a high-performance, data-intensive backend software that allows the advertising sector to intelligently target ads, is seeking a Senior Data Engineer.
As a Senior Data Engineer you will be responsible for the evolution of the data platform, developing the architecture and maintaining the data platform backlog. The systems generate vast amounts data, approximately 350GB per hour, distributed over many servers. You'll be solving complex technical challenges and finding new ways to use the data to improve the product and deliver more value.
A typical day could involve coding in Python or Scala; you'll design, build and manage large batch / streaming data processing products (e.g. Spark, Storm); design, build and maintain ETL pipelines; partnering with Data Scientists and other engineers to implement and optimise complex Machine Learning algorithms and help to support / mentor more junior Data Engineers in a collaborative environment.
Requirements:
Strong experience of solving complex, large scale data challenges
Strong Python (PySpark) and / or Scala coding skills for data intensive systems
Hands-on implementation and architectural familiarity with streaming data, relational and non-relational databases, orchestration frameworks (e.g. Luigi, Airflow or Oozie) and distributed processing technologies (e.g. Spark)
Familiar with cloud based services - AWS, GCP
Good working knowledge of Linux
Collaborative team member, keen to learn and share knowledge with others
As a Senior Data Engineer you will earn a competitive salary (to £90k) plus bonus and benefits.
Apply now or call for a confidential discussion about this Senior Data Engineer opportunity.