Job Title: Data Engineer
Closing date: 18 February 2019
STV is Scotland’s leading digital media brand, providing consumers with quality content on air, online and on demand. STV Player offers users the opportunity to enjoy a variety of their favourite programmes (from Coronation Street and Unforgotten to I’m a Celebrity and X-Factor) live or on catch-up. STV content is available across platforms including iOS, Android, YouView, Roku and most recently, Virgin Media. STV Digital is dedicated to making the best user experience a reality for digital users whilst at the same time harnessing the unique data generated to both improve the product and enhance commercial revenues.
As a Data Engineer, you will be working with multiple types of data generated within the organisation, from user generated event data, to marketing campaign information, to sales data. Your mission will be to constantly improve data quality, availability and usability across the organisation. On a day to day basis you will be working with our data team to collect, organise, and document our incoming and generated data. It will be your responsibility to develop quality, reliable solutions for collecting data and working to implement solutions to analyse and productionise developments generated within the team, both internally and for our products.
The primary responsibility of this role will be in designing and implementing fit for purpose, resilient and scalable data pipelines to ingest and transform data from its raw form in variety of sources, perform variety of analytics on it and prepare it for visualisations. You will be required to help us build systems for data analyses and visualisations, to make sense of data and to inform decisions clearly and truthfully. Candidates will ideally have a BI/ETL/analytics background.
• Ensure the data architecture is optimised for large dataset acquisition, analysis, storage, cleansing, transformation and reclamation
• Develop standards and methodologies for benchmarking, performance, evaluation, testing, data security and data privacy
• Design and build the data analytics platform that will enable single customer view to function
• Maintain awareness on current and new technologies and analyse their applicability to single customer view
• Recommend and set objectives for unified data management practices such as meta-data management, provenance management, governance, stewardship, data quality and lifecycle management
• Establish and enforce policies, procedures, standards, methodologies, and metrics for data quality, metadata management, and master data management
• Provide recommendations, technical direction and leadership for Hadoop and NoSQL technologies as part of the overall data architecture for which Data Engineer will have ultimate ownership
• Contribute to the development, review, and maintenance of requirements documents, technical design documents and functional specifications
Qualifications and Skills
• Ability to compose complex queries using SQL
• Keen interest in and capability of programming in Spark (Scala or Python)
• Demonstrated programming skills in Python for data wrangling (PySpark)
• Knowledge of the following AWS services (S3, EC2, Lambda, RDS, and Data Pipelines)
• Comfortable working with shell scripting (bash)
• Independent, motivated, collaborative, data lover
• Collaborating with clients to build supportive and trusting relationships
• Making complex concepts simple and easy to understand
• A love for metrics and efficiency
• Ability to quickly grasp other languages
• Knowledge of visualisation tools such as Tableau, Qlik, and Microsoft BI etc.
• Excellent communication and interpersonal skills to work collaboratively with the team and build effective working relationships with internal and external stakeholders.
• A team player who supports and shares knowledge with wider Data team members.
• Organised, eye for quality, attention to detail and high level of accuracy in all aspects of work output.
• Strong self starter, results and solutions driven approach required.
• Keen learner with a passion for staying up to date with latest trends / developments in Data Engineering