Senior Data Engineer - Pipelining


Premium Job From JLR

Recruiter

JLR

Listed on

1st September 2021

Location

Jaguar Land Rover Manchester

Type

Permanent

Start Date

2021-09-01

This job has now expired please search on the home page to find live IT Jobs.

A leading premium automotive business and leading to move towards electric autonomous vehicles.

THE OPPORTUNITY
We are looking for a Senior Big Data Engineer with proven experience of maintaining data pipelines using a broad range of data pipeline runners that handle high throughput with low latency in production, supporting pipeline lifecycles with customers, supporting other big data engineers, and advising other technical staff in big data engineering best practices.

This role provides a rare opportunity to support and grow an entirely new data capability within Jaguar Land Rover's Global Digital Development team. You will form an essential part of a collaborative, iterative and agile process within a team of designers, software, DevOps and quality engineers helping deliver best in class digital solutions to the business and its customers.

You will be responsible for:
* Helping customers within JLR to create, run, and manage their own data pipelines using our pipeline-as-a-service product.
* Observing, maintaining, and fine-tuning customer's pipelines in production.
* Helping feed the product backlog for the pipeline-as-a-service product via your interactions with customers and your knowledge of pipeline best practices.
* Mentoring other big data engineers and interacting with other teams in the pipelines-as-a-service group to help deliver customer satisfaction and enable the product group to become world-leaders in this novel space.


Do you thrive in an exciting, dynamic and creative start-up environment, but also enjoy the security, resources and influence of an established and successful organisation? Would you excel within a world-class and passionate team that loves what they do? Are you looking for an influential role that can truly make a difference?

Reporting directly to the Principal Software Engineer and working closely with the Lead Big Data Engineer on your team, you will be a valued member of the Global Digital Development team, contributing to the strategic direction of the data technology and solutions delivered.

The Senior Data Developer is responsible for the following:

* Administering running pipelines via the Google Cloud Dataflow pipeline runner
* Interfacing with pipeline ingress and egress mediums (Google Cloud Storage, Google Pub/Sub, Kafka via Confluent)
* Helping customers to design and implement data pipelines (batch/streaming) using the pipelines-as-a-service product.
* Helping enterprise architects, software developers, and other big data engineers understand big data engineering concepts and effective usage of data pipelines by encouraging and teaching best practices.
* Effectively fine-tuning the performance of running pipelines in production.
* Advise on how best to achieve the scale required by the customer-base of the product.
* Being able to identify where pipeline running costs can be reduced without sacrificing customer satisfaction.
* Consistently delivering to challenging business objectives and adhering to Service Level Agreements with commercial implications for success and failure
* Demonstrating strong problem-solving and analytical skills both individually, and as part of a group
* Introducing and driving the usage of open source libraries and tools as appropriate to optimise delivery timescales and quality of the pipelines-as-a-service product.


WHAT YOU WILL NEED
Essential Skills & Experience:

* Strong experience of running, observing, and fine-tuning big data pipelines in production environments using Google Cloud Dataflow.
* Strong knowledge of best practices for, and administering, streaming technologies including Kafka, Confluent, Apache Beam, Apache Spark, and Google Pub/Sub
* Experience in object-orientated programming, ideally using Java.
* Experience with Amazon Web Services and Google Cloud Platform
* Ability to collaborate with software engineers, platform-reliability engineers, product owners, agile delivery leads, architects, and customers in cross-functional teams.
* Willingness and ability to contribute to process improvement initiatives.
* Ability to coach and mentor other big data engineers to achieve high quality pipelines.
* Excellent written and verbal communication skills


Desirable:
* Experience of pairing to undertake role.
* Experience of working in an Agile/XP/Scrum team and familiarity with associated techniques.
* Awareness of DevOps culture and associated practices: Continuous Integration and Delivery, Infrastructure Automation, Automated Monitoring and Analytics

You are currently using an outdated browser.

Please consider using a modern browser such as one listed below: