Hadoop Developer
Recruiter
SidTech
Listed on
6th February 2020
Location
Bristol
Type
Contract
Start Date
2020-02-06 00:00:00
This job has now expired please search on the home page to find live IT Jobs.
Hadoop DeveloperBackground The IFRS17 project is utilising a Hadoop big data layer that processes insurance data to produce reports for the new regulatory standard. Objective Under the supervision of the team (a data wrangler and two other developers), the developer will be responsible to:
- write a Spark tool, in Scala, for converting XML data into Apache Hive tables
- write code that process Insurance data to create ‘events’ that will be used in the reporting layer
- write and maintaining unit and acceptance tests (in TDD/BDD styles)
- support the tool once it is released Requirement Essential
· Strong Scala experience
· Strong Spark experience, preferably working with Spark SQL (datasets / dataframes)
· Worked with structured data before, particularly XML.
· Worked in an agile team before
· Writing and maintaining unit tests / acceptances tests (TDD / BDD)
· Experience with Git or other VCS tools
· Some experience with dev ops tools such as Jenkins, Nexus, Urbancode.
· JIRA/Confluence/Agile-Scrum-Kanban methodology
· Experience with Linux shell
Desirable
· Experience with Scala libraries such as Cats, Shapeless or Frameless.
· Experience parsing structured data using Spark SQL
· Experience with Maven or other build tools (SBT, Gradle, etc)
· Knowledge and experience of pure functional programming, in Scala or other programming languages (Haskell, Elm, etc) Capability · Will take direction from Lead Hadoop developers
· Will liaise with Data Wranglers/Business Analysts, Business Subject Matter experts and other staff
· Ability to work under own direction, self-starter, proactive approach to identifying work packages and working with the team to identify solutions
· Strong communication skills - Conference calls, huddles, written and verbal skills Business Experience At least 2 years’ experience in the following
· Financial systems/projects & Data processing projects
