Senior Data Engineer


Premium Job From Nigel Frank International

Recruiter

Nigel Frank International

Listed on

21st August 2021

Location

Scotland

Salary/Rate

Upto £65000

Type

Permanent

Start Date

ASAP

This job has now expired please search on the home page to find live IT Jobs.

Senior Data Engineer - FULLY REMOTEUp to £65k DOEMy client are seeking a talented individual to help increase their delivery, managing a continuous integration and deployment tools, helping to define infrastructure as code, and with estimating system requirements, as well as monitoring smooth operations.You will work with external and internal stakeholders as well as team members when capturing and documenting IT requirements.You will show attention to detail and show strong problem solving skills when helping to spec out projects and infrastructure.You will help with the development of data pipelines, setting up data lake and data warehousing infrastructure.You will help creating new integrations with third party providers and external data sources.You will help with the development of dashboards and other reporting tools.You will be comfortable working to Agile principles and development methodologies. Your SkillsRequiredExcellent knowledge of PythonYou will know common data query languages for relational and document databases.Experience working with version control tools, understand the importance of code reviews and coding standards).Experience with Messaging queues and event driven development (eg. Rabbit MQ, SQS).Experience setting up data pipelines, data extraction, transformation and load processes.Experience developing REST APIs.Understanding of common cloud metrics and alerting within AWS.DesirableExperience with Apache Spark and Databricks.Experience using the AWS eco system, especially VPC, EC2, RDS and Redshift and the AWS Cloud Development Kit, Lambda.Experience with workflow management tools (eg. Airflow, Luigi).Experience working with continuous integration, CI/CD processes.Experience with orchestration tools (eg. Cloud formation, Kubernetes, etc). To apply or for more info contact [email protected] or call 0191 338 7638.

You are currently using an outdated browser.

Please consider using a modern browser such as one listed below: