Senior/Lead Data Engineer - GCP


Premium Job From Nigel Frank International

Recruiter

Nigel Frank International

Listed on

23rd October 2023

Location

London

Salary/Rate

£65000 - £70000

Type

Permanent

Start Date

ASAP

This job has now expired please search on the home page to find live IT Jobs.

What job we expect to do:We are looking for a GCP Lead Engineer to join our Data Insights and Analytics team. The role will be responsible for building new data pipelines, maintaining existing ETL pipelines, and optimizing data flows using the GCP cloud stack. The ideal candidate will be an experienced resource with strong experience in building data pipelines, data products and platforms from scratch as well as maintaining and fine tuning existing setups. The data engineer will need to support Business Analysts, Technical team members and other stakeholders. He/she must be self-directed and comfortable supporting the data needs of multiple teams, systems and products. Skills we expect to bring::AdvancedGCP knowledge, experience working on and migrating data products from on-prem to GCPExperienceof leading teams to deliver projectsExperiencebuilding and optimizing 'big data' data pipelines, architectures and data sets using GCP servicesExperiencebuilding real-time data pipelines including unstructured datasetsBuild processes supporting data transformation, data structures as well ETL, cost, space optimizationStrong experience in Data Ingestion and Storage including BigQuery, GCS, Dataflow, Datastream, Dataform, AirflowExperience with Big data Tools: Hadoop Spark, KafkaExperience with Stream processing systems: Spark-streaming, KafkaStrong problem solving, quantitative and analytical abilitiesExcellent communication, collaboration and delegation

What Tools and Technologies do we expect you to know? We understand one cannot be master of all.

SQL using Teradata, DB2, Oracle, etc.Cloud native data warehouse like BigQueryPython programming including Pandas, NumpyGood experience of ETL, Data Modelling, Data Warehousing, etc.GCP - BigQuery, GCS, Dataflow, Datastream, Dataform, Airflow

You are currently using an outdated browser.

Please consider using a modern browser such as one listed below: