Big Data Engineer

This job has now expired please search on the home page to find live IT Jobs.

I am working with a Central London based Financial organisation that have had 3 - 4 significant projects land in the last month, and are therefore looking to hire a contract Big Data Engineer for 3 - 6 months. They are well established and having a really impressive data team consisting of mainly PhD level members.Duration: 6 MonthsDay Rate: £500 - £600 per dayDay to day role:

'€¢Responsible for supporting the developers'€¢Expanding and optimising the data pipelines on Hadoop, PostgreSQL and Spark to deliver usable data to business requirements'€¢Deploy to, work on and manage AWS especially EMR and S3

Essential Skills:

'€¢Management of Hadoop cluster, with all included services and an ability to solve any ongoing issues with its operation'€¢Proficiency with Hadoop v2, MapReduce, HDFS'€¢Experience with building stream-processing systems'€¢Good knowledge of Big Data querying tools, such as Pig, Hive, and Impala'€¢Experience with NoSQL databases, such as HBase, MongoDB'€¢Knowledge of various ETL techniques and frameworks,'€¢Experience with various messaging systems, such as Kafka'€¢Experience with Big Data ML toolkits, such as Dask, SparkML, or H2O'€¢Experience with Google Cloud Platform/Hortonworks'€¢Experience in Python, particularly the scientific stack and Anaconda'€¢Able to integrate multiple large data sources and databases into one system.

The role has just opened this week, and the client are looking to interview from WC 2nd July 2018.If this role seems like it is of interest to you and you fit the requirements for the role please apply today by sending your CV to or call Lydia on for further details. I am also working several other contract roles across London and South East of UK, so if you are on the lookout for a new Big Data role please apply anyway.