Big Data Engineer


Premium Job From Deerfoot

Recruiter

Deerfoot

Listed on

13th November 2018

Location

Dubai

Salary/Rate

£5000 - £8000

Type

Contract

Start Date

ASAP

This job has now expired please search on the home page to find live IT Jobs.

Contract Big Data / Cloud Engineer - 20,000 - 40,000 AED Per / Month (£5,000 - £8,000) - Dubai - 12 months Initially - Hadoop, Azure, Cloudera

Long term contract

Dubai

Are you looking to gain commercial exposure to an international client? An opportunity to work for one of the most prestigious organisations in the world based in Dubai has arisen for a Big Data / Cloud Engineer

A Big Data engineer builds large-scale data processing systems, is an expert in data modelling and processing solutions and should be able to work with the latest Big Data and NoSQL database technologies. Because we work on the cutting edge of a lot of technologies, we need someone who is a creative problem solver, resourceful in getting things done, and productive working independently or collaboratively.

Key Skills:

*Having 3+ years exposure on the Cloudera Manager, Cloudera Navigator, Hue/Hive, Scala, Spark, Shell Scripting, Apache Pig, Python and Ooziee workflows.

*Having 2+ years working experience in Kafka, Flume and Spark Streaming.

*Having exposure on ADLS, Azure Data Factory and SnapLogic.

*Good understanding in XSD, XML and JSON schemas.

*Integrating Hadoop with Oracle and NoSQL databases.

*Experience in DevOps tools (CI/CD).

*Automated workflow migration from on premise to cloud experience would be added advantage

*Use technical tools or native programming languages, to find, identify and diagnose solution defects based on data models or data extracts. Write advanced SQL queries to verify the integrity of databases including data transformations

Required Experience:

*Minimum 5+ years of Developing and testing experience in the Apache Hadoop framework, Azure cloud, Lambda architecture and Data Lake.

*Experience in Big Data and batch/real-time ingestion and analytical solutions leveraging transformational technologies.

*Experience in Java, spring, Elasticsearch and Cassandra.

*Understand the source systems data, gather and able to transfer the data into HDFS as RAW and Decomposed layer (including writing Oozie workflow, coordinators, Sqoop and Flume).

*Ability to identify Unit testing.

*Experience and knowledge of a wide variety of testing methods and tools covering functional, performance, regression, system integration, smoke testing of Business Intelligence, ETL, BigData ( Ingestion, Modeling, Serving layer) and Business Analytical landscape.

We are keen to build strong relationships with IT consultants based on providing a transparent and supportive service. Now in our 18th year we pay promptly and are a stable contracting partner in good financial health. Deerfoot IT: Est. 1997. REC member. ISO certified. *Each time we send a CV to a recruiting client we donate £1 to The Born Free Foundation (charity no. 1070906). We never send your CV without your email authorisation.

You are currently using an outdated browser.

Please consider using a modern browser such as one listed below: