Data Engineer

This job has now expired please search on the home page to find live IT Jobs.

Data Engineer£80,000 - £90,000Fully Remote We are recruiting for a Data Engineer who will help propose and implement solutions primarily based around AWS, including designing and implementing ETL tasks, identifying key data relationships. The role will involve working with colleagues, both IT and business, to help define the longer term strategy.The role will also involve building data pipelines that allow data to be used by cross functional teams, including data scientists for ML. Data analysis of source data structures is required to enable the most efficient loading and transformation patterns to be designed. There will be a focus on automating tasks and continuous integration based delivery.Essential Skills:Cloud environments, AWS as strong preference.ETL, ELT, data migration and cleansing.Data Lakes, data analytics and working with large data setsRelational databases.Agile working.The ability to work independently yet seek advice where required.Committed to delivering excellence for customers.Essential experience:Hands on experience with AWS, building and configuring environments.Familiarity with variety of Databases, incl. structured RDBMS.Experience in analysing and cleansing data using a variety of tools and techniques.Experience in working with a variety of data formats, JSON, XML, CSV, Parquet etc.Experience with building and maintaining data dictionaries / meta-data.Experience of Linux and cloud environments.Using patterns for cloud data warehouse solution (e.g. ELT).Desirable experience:Hands on experience with AWS Glue and S3.Familiarity with AWS data lake related components.Familiarity with large data techniques (Hadoop, MapReduce, Spark etc.)Familiarity with providing data via a micro service API.Experience with other public cloud data lake.AWS Certifications (particularly Solution Architect Associate and Big Data Speciality).Machine Learning.Data Visualisation Technologies (e.g. Amazon QuickSight, Tableau, Looker, QlikSense).Familiarity with ETL tools (Matillion, Alteryx, SSIS).