Senior Data Engineer - Hybrid
Recruiter
BCT Resourcing
Listed on
5th March
Location
City of London
Type
Permanent
Start Date
2024-03-05 16:11:44
This job has now expired please search on the home page to find live IT Jobs.
Senior Data Engineer - AWSLondon / Hybrid
Were shaping the future of financial services by collaborating with business partners to revolutionise client investment experiences. As part of the Engineering team, youll play a key role in designing, constructing, testing, and managing the data platform, surrounded by talented colleagues in a collaborative environment that values diverse perspectives and fosters continuous learning and growth.
You will be responsible for:
* Contributing to the platforms architectural design
* Development and testing of the integration, modelling, data persistence and querying tools and analytical systems
* Data pipeline implementation, maintenance and testing
* Metadata management processes and tools
* Monitoring the overall performance and stability of the pipelines
* Implementing data curation, metadata management and data quality tooling
* Engaging with our business and technology partners to ensure that the platform fully meets the firms needs
Preferred Technical experience/skills:
* 7+ years of experience working as a Java/Python developer.
* Strong understanding of AWS ecosystems like Lambdas, step functions and ECS services.
* Experience with data stack technologies, such as Apache Iceberg & Spark. Exposure to Apache Airflow, Prefect, Dagster, DBT.
* Expertise in data analysis with exposure to data services (such as Glue, Lake Formation, EMR, EventBridge, Athena, etc.) & metadata management tools (such as Amundsen, Atlas, DataHub, OpenDataDiscovery, Marquez, etc.),
* Experience on RDBMS like PostgreSQL would be a plus.
* Experience building modern enterprise-wide data and analytics systems in a financial services organisation with an understanding of the asset management business and/or financial markets
