Lead Data Engineer

This job has now expired please search on the home page to find live IT Jobs.

The Digital Architecture and Engineering team within RSSB is a centre of excellence, tasked with servicing the growing need for integrated data systems and data driven insights.  This covers a broad range of rail industry focus areas, including health and wellbeing performance management, safety risk modelling, and sustainability monitoring.  The team also develop internal business intelligence and reporting solutions.

The team primarily work with the Azure cloud platform, utilising Postgres for data warehousing and Power BI for visualisation. Our Data Engineering pipelines involve Python and SQL development.  We are regularly looking for opportunities to make use of the latest technologies and achieve best practice development with continuous integration and continuous delivery.

As the Lead Data Engineer, you will support project delivery, working as part of Agile delivery teams. You will work both as a technical resource and engage with business and industry stakeholders, to understand requirements and deliver technical solutions.  From a technical perspective, you will build pipelines and simple APIs to load data into the RSSB reporting platform, develop scripts and ETL processes to transform data, and develop data warehouse and Power BI models for analysis and reporting.

Responsabilities

Work with stakeholders to build effective working relationships, understand their data requirements and shape technical delivery.

Break down complex projects into simple components that can be effectively built and maintained by less experienced data engineers. Setting clear goals for success and clarifying technical ownership and decision making.

Design and implement well-architected data engineering frameworks and systems.

Analyse and resolve tactical issues across the data engineering pipeline, collaborating with senior internal and external stakeholders to solve problems and resolve technical debates.

Mentor other engineers and managers on technical leadership, including strategy, collaboration, influence, and execution.

Maintain awareness of changes in the organisation, anticipating and preventing obstacles that may hinder the team from meeting its objectives.

Ensure there is deep technical knowledge of the applications, services, systems, and frameworks within the team, and this is appropriately documented and shared amongst the team.

Contribute to defining the strategic direction, planning the roadmap, escalating issues, and analysing feedback to ensure team success.

Develop and maintain the structure of the reporting database and related data flows and transformations, to combine data from multiple sources for delivering data insights and business intelligence.

Ensure technical solutions support the accessibility, retrievability, security, quality, retention and ethical handling of specific sets of data.

Examine data requirements, and apply data analysis, design, modelling and quality assurance techniques to modify or maintain data structures and associated data transformation components.

Investigate ways to extract additional value from existing information assets, including the application of new tools and techniques to make data insights more accessible.

Ensure that agreed standards and tools are applied to achieve well-engineered outcomes, including incorporating monitoring and logging functionality into all aspects of the data platform.

Participate in reviews of own and colleagues' work to ensure these components are reliable and maintainable. Optimise code to ensure processes perform optimally and reliably.

Qualifications

Experience of working with a diverse range of systems and applying complex data engineering techniques to resolve complex technical problems; demonstrate a positive disposition towards problem solving.

Experience exercising substantial personal responsibility and autonomy, possessing effective organisational skills and the ability to work independently and with others.

Experience influencing customers, supplier and peers on data and business intelligence development and processes.

A commitment to self-development and continuous improvement, maintaining an awareness of developing data engineering practices and their application.

Ability to clearly communicate complex technical information to both technical and non-technical audiences.

Experience and knowledge of data analysis, data modelling and quality assurance techniques.

Experience designing, coding, testing, and correcting complex SQL models and Python applications.

System/Data integration experience, including application programming interface design, and familiarity with web-oriented architecture techniques.

Experience of end-to-end project delivery, including managing project plans and reporting.

Experience of building and maintaining effective customer relationships, at both user and management levels.

Ability to develop capabilities and lead a team, establishing performance objectives and monitoring outcomes.

Ability to work through others and lift the skill and expertise of colleagues.

A commitment to RSSB's values and customer service.

Early applications are encouraged, as we may close this vacancy before the advertised deadline if sufficient numbers of applications are received.