Data Engineer - Scala & Spark - Media Co.


£500 a day

My client is an award winning, cutting edge, global, independent entertainment services company that are the market leaders in their field. Technology is at the forefront of everything the company does and underpins all the services provided.

Your will be working for a company with 500 staff with three core global offices. You will be working in a data engineering team of 3 people including the manager.

This is a data engineering role and not a BI type position. There is a separate Data Science team here that is involved with the R&D and building of new ideas. Then you in the Data Engineering team will build the solutions that are rolled out into production of our client's software product.

You'll have the ability to identify solutions to the problems that big data creates. You will work the whole development lifecycle: requirements gathering, specification, development, building, testing and deployment of new analytics and data capabilities, interfacing with our existing systems as well as supporting already deployed solutions.

Key Skills and Experience Required:

Scala - Solid knowledge needed
Spark / Apache Spark knowledge needed
Data Engineering / Data services / Machine learning
Familiarity with relational databases - SQL/NoSQL, Redshift
Experience of creating data APIs
Hadoop knowledge would be beneficial
Amazon Web Services very beneficial
All things Unix / Linux
Shell scripting (Bash)
Use of Jira and Git and ability to operate in a developing AGILE environment
Problem-solving - comfortable dealing with high volume of unstructured and varied data

Some duties:

Define and build solutions that will determine data solutions
Bring process to data engineering function and suggest improvement to approach technology and tools
Carefully plan workload - identify and resolve any risk, issues, and dependencies and fix problems as they occur
Build infrastructure in the cloud for development and data hosting
Diagnose and fix performance issues
Manage the development lifecycle to deployment into production
Support the use of Airflow (Python-based workflow management tool)
Support colleagues and share knowledge - present project progress in group meetings, up skill and coach others in technology and techniques used

VERY GOOD COMPANY HERE. Contract extensions likely. APPLY ASAP.

Dress down company culture.

Data Engineer - Scala & Spark - Media Co.

Posted by London Office, 13 Oct