The data engineering team sits alongside our Data Science, MI & Insight and Data Ops teams.
Data Science focuses on data R&D, evolving our data products and creating new ones. MI & Insight focuses on the presentation of data in a consumable and appropriate manner to enable data led decisions. Data Ops is responsible for the deployment of our data products and the management of the team’s infrastructure.
Data Engineering is the glue that sticks the teams together, enabling products created by data science to be deployed into production, results from those products being made available to MI & Insight for presentation and working with Data Ops to ensure that appropriate pipelines are built and that our production deployments are working correctly.
This role is part of the Data Engineering team and will have a high degree of autonomy and ability to make technical decisions. Using previous AWS & Python development experience it will help create, deploy & maintain our data products in a cloud first world.
- Responsible for the creation, deployment & maintenance of all of our data products these include (but are not limited to)
- Crash Detection
- Driver Behaviour Scoring
- Distracted Driving Detection
- Driver Fingerprinting
- Geocoding & Data Enrichment
- Writing effective, resilient, scalable & secure code
- Focus on always leaving code in a better place than you found it
- Developing & maintaining appropriate testing
- Debugging issues with existing systems & producing recommendations to fix
- Producing solution designs for new data products & gaining buy in for their delivery
- Coordinate with stakeholders to understand requirements
- Manage own time and communicate any blockers as soon as possible to wider team
- Significant hands-on experience of developing in Python
- Significant hands-on experience of deploying large volume platforms in AWS with knowledge of the following tools & technologies
- Lambda & Step Functions
- Athena/RDS/Dynamo DB/ElastiCache/S3
- Experience of Docker a must
- Hands on experience working in a continuous delivery environment
- Ability to test appropriately and focus on quality
- Experience of working with data science teams and Python data science packages such as Pandas/Dask/SKLearn/Keras very desirable
- Understanding of data modelling & designing performant data structures advantageous
- Comfortable communicating very technical concepts to a non-technical audience and where necessary gaining their support & buy in on decisions
- A mentality to always try to do better and never leave code in a worse state than when you found it
- Ability to turn high level requirements into a plan including specific technical deliverables