Online since 1999 | 9,185 IT Jobs Live NOW

Senior Data Engineer

Premium Job From Nigel Frank International
Recruiter: Nigel Frank International
Listed on: 9th July
Location: Scotland
Salary/Rate: Upto £65000
Type: Permanent
Start Date: ASAP

This job has now expired please search on the home page to find live IT Jobs.

Senior Data Engineer - FULLY REMOTEUp to £65k DOEMy client are seeking a talented individual to help increase their delivery, managing a continuous integration and deployment tools, helping to define infrastructure as code, and with estimating system requirements, as well as monitoring smooth operations.You will work with external and internal stakeholders as well as team members when capturing and documenting IT requirements.You will show attention to detail and show strong problem solving skills when helping to spec out projects and infrastructure.You will help with the development of data pipelines, setting up data lake and data warehousing infrastructure.You will help creating new integrations with third party providers and external data sources.You will help with the development of dashboards and other reporting tools.You will be comfortable working to Agile principles and development methodologies. Your SkillsRequiredExcellent knowledge of PythonYou will know common data query languages for relational and document databases.Experience working with version control tools, understand the importance of code reviews and coding standards).Experience with Messaging queues and event driven development (eg. Rabbit MQ, SQS).Experience setting up data pipelines, data extraction, transformation and load processes.Experience developing REST APIs.Understanding of common cloud metrics and alerting within AWS.DesirableExperience with Apache Spark and Databricks.Experience using the AWS eco system, especially VPC, EC2, RDS and Redshift and the AWS Cloud Development Kit, Lambda.Experience with workflow management tools (eg. Airflow, Luigi).Experience working with continuous integration, CI/CD processes.Experience with orchestration tools (eg. Cloud formation, Kubernetes, etc). To apply or for more info contact or call 0191 338 7638.