Data Engineer - Remote

This job has now expired please search on the home page to find live IT Jobs.

Data Engineer - Mid/senior£45-65K, DOE, Negotiable + BenefitsFully remote options The Team and RoleOur teams persistently works on a backlog of data sources and integration's with third party vendors and suppliers to create a data warehouse within the AWS eco system.The ResponsibilitiesWe are seeking a talented individual to help us increase our pace of delivery, managing a continuous integration and deployment tools, helping us to define infrastructure as code, and helping us with estimating system requirements, as well as monitoring smooth operations.You will collaborate with external and internal stakeholders as well as team members when capturing and documenting IT requirements.You will show attention to detail and show strong problem solving skills when helping to spec out projects and infrastructure.You will help with the development of data pipelines, setting up data lake and data warehousing infrastructure.You will help creating new integration's with third party providers and external data sources.You will help with the development of dashboards and other reporting tools.You will be comfortable working to Agile principles and development methodologies. Your Skills:Required:PythonCommon data query languages for relational and document databases.Setting up data pipelines, data extraction, transformation and load processes.Version control tools, understand the importance of code reviews and coding standards.Messaging queues and event driven development (eg. Rabbit MQ, SQS).Developing REST APIs.Understanding of common cloud metrics.Desirable:Experience with Apache Spark and Databricks.Experience using AWS, such as VPC, EC2, RDS and Redshift and Lambda.Experience working with continuous integration, CI/CD processes.Experience with orchestration tools such as Cloud formation, Kubernetes.Experience with workflow management tools such as Airflow, Luigi.To apply for this role, please send an up-to-date CV to [email protected] or call 0191 300 9229 for a confidential chat. LinkedIn = https://www.linkedin.com/in/adam-quigley-945824203/Jefferson Frank is the Amazon Web Services (AWS) recruiter of choice. We work with organizations worldwide to find and deliver the best AWS professionals on the planet. Backed by private equity firm TPG Growth, we have a proven track record servicing the AWS recruitment market and, to date, have worked with over 30,000 organizations globally.