Senior Data Engineer


Premium Job From IC Resources

Recruiter

IC Resources

Listed on

4th January 2020

Location

Cambridge

Salary/Rate

£45000 - £59000

Type

Permanent

Start Date

ASAP

This job has now expired please search on the home page to find live IT Jobs.

Senior Data Engineer

Job Description

Are you a passionate software developer with a keen eye for detail and engineering elegance? Do you love solving problems, innovating and developing ideas? Are you proud of past work where you have delivered high quality, performant, and maintainable software solutions?

Operations Research/ Decision Analytics Team (OR/DA) are looking to expand our team with a talented and experienced Senior Data Engineer to work on building automated data pipelines for the Operations Research and Decision Analytics function. We are part of a centralised business group which runs and delivers the platforms, tools, infrastructures and workflows that are used by the Arm product development teams.

The OR/DA team works with data to add business value by applying a mixture of Data Engineering, Data Visualisation and Data Science disciplines to a variety of complex engineering problems. We extract information from large data sets to enable data driven decisions and improved efficiency and effectiveness of use of engineering platforms and infrastructure

You will be an experienced software engineer with experience of Spark, Python, databases and/ or equivalent data engineering tools, and keen to pursue the problems of extracting knowledge from data and driven to understand the core 'stories' that data can tell. You will help drive efficiency and effectiveness improvements by providing the means to make data driven decisions.

Responsibilities

* You will take ownership for the development and deployment of our ETL data engineering process, that allows relevant data to be centrally collected, aggregated, and processed

* You will work with engineering and IT functions to ensure sufficient relevant data is being gathered to provide the best information for engineering platforms like HPC, Storage, EDA, Emulation, FPGA's..

* You will have continuous communication with the organisation for changing priorities and improvements needed for development of the service in the area of data engineering

Your tasks will typically include

* ETL Code Development Data source exploration, extract, transform and load engineering data from various sources for visualisation

* Build and deploy ETL workflows for continuous data delivery to the dashboards making sure it is robust and scalable

* Follow software engineering standards Testing analytics products for robustness Source Control, Code Testing, Verify Code Logic and Data integrity, Debugging and Review of Code and Data anomaly.

* Enhance data collection and data processing methodologies.

* Collaborate with the other team members who use various statistical and Machine Learning models to build predictive analytical products for our Engineering and Platform Operation teams

Job Requirements

Required Skills And Qualifications

* A good university degree in computer science/software engineering or equivalent life experience and knowledge.

* You will consider yourself to have extensive experience and a track history of delivering complex software projects on schedule and within budget

* A validated proficiency in one or more programming languages such as Scala JavaScript, Java, Python, C#, Go, Perl and a passion to learn more!

* Sound understanding of software development process and practices such as coding standards, code reviews and version control systems for software build and test solutions such as Jenkins, TeamCity, Bamboo

We would be even more impressed if you have had experience/ passionate about the following

* Exposure to large scale clustered compute environments (LSF) and Storage Architecture with some exposure to cloud services

* Knowledge of SQL, NoSQL, Exasol, Linux, Docker, Mesos, etc.

* NoSQL distributed databases, such as Scylla, Cassandra

* Stream processing framework for distributed, always-available data streaming applications like Kafka, Flink

* Data analysis and visualisation using Hadoop, Elasticsearch, Kibana, Tableau D3.js, etc.

* An understanding of virtualisation and cloud technologies such as Docker, Vagrant, OpenStack, Kubernetes etc.

* Knowledge of working with cloud services like AWS (S3, Kinesis, EMR, Redshift, Glue, Lambda etc..)

* Knowledge of data preparation techniques for machine learning

Additionally, you will make the biggest impact in this role if you are

* Self-motivated

* Highly inventive

* Hardworking

* Passionate about making a difference

* Ability to work independently and in teams; working in collaboration with engineers/ scientists and business partners.

* Excellent communication skills; able to rationalise your approach and influence

Arm has a responsibility to ensure that all employees are eligible to live and work in the UK.

You are currently using an outdated browser.

Please consider using a modern browser such as one listed below: