Data Engineer

This job has now expired please search on the home page to find live IT Jobs.

Job Title: Data Engineer Location: Glasgow Closing date: 18 February 2019 STV is Scotland's leading digital media brand, providing consumers with quality content on air, online and on demand. STV Player offers users the opportunity to enjoy a variety of their favourite programmes (from Coronation Street and Unforgotten to I'm a Celebrity and X-Factor) live or on catch-up. STV content is available across platforms including iOS, Android, YouView, Roku and most recently, Virgin Media. STV Digital is dedicated to making the best user experience a reality for digital users whilst at the same time harnessing the unique data generated to both improve the product and enhance commercial revenues. As a Data Engineer, you will be working with multiple types of data generated within the organisation, from user generated event data, to marketing campaign information, to sales data. Your mission will be to constantly improve data quality, availability and usability across the organisation. On a day to day basis you will be working with our data team to collect, organise, and document our incoming and generated data. It will be your responsibility to develop quality, reliable solutions for collecting data and working to implement solutions to analyse and productionise developments generated within the team, both internally and for our products. The primary responsibility of this role will be in designing and implementing fit for purpose, resilient and scalable data pipelines to ingest and transform data from its raw form in variety of sources, perform variety of analytics on it and prepare it for visualisations. You will be required to help us build systems for data analyses and visualisations, to make sense of data and to inform decisions clearly and truthfully. Candidates will ideally have a BI/ETL/analytics background. Responsibilities                                Ensure the data architecture is optimised for large dataset acquisition, analysis, storage, cleansing, transformation and reclamation                                Develop standards and methodologies for benchmarking, performance, evaluation, testing, data security and data privacy                                Design and build the data analytics platform that will enable single customer view to function                                Maintain awareness on current and new technologies and analyse their applicability to single customer view                                Recommend and set objectives for unified data management practices such as meta-data management, provenance management, governance, stewardship, data quality and lifecycle management                                Establish and enforce policies, procedures, standards, methodologies, and metrics for data quality, metadata management, and master data management                                Provide recommendations, technical direction and leadership for Hadoop and NoSQL technologies as part of the overall data architecture for which Data Engineer will have ultimate ownership                                Contribute to the development, review, and maintenance of requirements documents, technical design documents and functional specifications Qualifications and Skills                                Ability to compose complex queries using SQL                                Keen interest in and capability of programming in Spark (Scala or Python)                                Demonstrated programming skills in Python for data wrangling (PySpark)                                Knowledge of the following AWS services (S3, EC2, Lambda, RDS, and Data Pipelines)                                Comfortable working with shell scripting (bash)                                Independent, motivated, collaborative, data lover                                Collaborating with clients to build supportive and trusting relationships                                Making complex concepts simple and easy to understand                                A love for metrics and efficiency                                Ability to quickly grasp other languages                                Knowledge of visualisation tools such as Tableau, Qlik, and Microsoft BI etc. Personal Qualities                                Excellent communication and interpersonal skills to work collaboratively with the team and build effective working relationships with internal and external stakeholders.                                A team player who supports and shares knowledge with wider Data team members.                                Organised, eye for quality, attention to detail and high level of accuracy in all aspects of work output.                                Strong self starter, results and solutions driven approach required.                                Keen learner with a passion for staying up to date with latest trends / developments in Data Engineering