Senior Data Engineer

This job has now expired please search on the home page to find live IT Jobs.

Senior Data Engineer£55,000 - £84,000 + Excellent pensionCentral London Data Engineer / Data Developer / Data / Data Science / Big Data / Azure / Hadoop / Hive / SQL / PythonWorking for a Government funded regulatory body who are using cutting edge data focused technology, the successful candidate will have the opportunity to work in a well funded and talented Data Services team.The Senior Data Engineer will responsible for the construction of data products for the business delivering new functionality to the Data Services platform. You will engage in discussion with technical design leads and other technical staff, recommending options for implementing technical infrastructure and future data products.Responsibilities:Facilitate the end-to-end data handling approach of Data Pieplines through the Data Services platform (defining and communicating technical needs for data collection, ingestion, transformation and calculation)Assure that your work and the work of colleagues meets governing standardsEnsure principles of privacy and security are assessed and designed into your data solutionsEncourage teamwork and communication, especially between Data Engineers, Quality Assurance and Subject Matter ExpertsFollow Agile methodologiesBe knowledgeable about the portfolio of data journeysProvide succinct documentation to support knowledge retention and trainingBe supportive and open to junior members of the team Skills / Abilities:Advanced knowledge of software design principles Data development processData integration designProgramming and buildTechnical understandingKnowledge of data management, handling and processing Data QualityData PreparationData CleansingData ModellingExperience working with and integrating into Business Intelligence tools Hands on experience in platforms such as Azure and the Hadoop ecosystem (HIVE, HDFS and Apache spark)Advanced competency with Python and SQLKnowledge of Linux