Senior Data Engineer - DBT - G7
Government Digital & Data -
The Department for Business and Trade (DBT) has a clear mission - to grow the economy. Our role is to help businesses invest, grow and export to create jobs and opportunities right across the country. We do this in three ways. Firstly, we help to build a strong, competitive business environment, where consumers are protected and companies rewarded for treating their employees properly.Secondly, we open international markets and ensure resilient supply chains. This can be through Free Trade Agreements, trade facilitation and multilateral agreements.Finally, we work in partnership with businesses every day, providing advance, finance and deal-making support to those looking to start up, invest, export and grow.The Digital, Data and Technology (DDaT) directorate develops and operates tools and services to support us in this mission.
About the team
To enable the work of the department you will join our Digital Data and Technology (DDaT) team of over 200 colleagues working across the full spectrum of the DDaT specialisms, in a modern, user-centric, agile way. You will join the team that build and operate innovative tools and services such as great.gov.uk that support businesses in the UK and overseas. Our public-facing services are visited millions of times each year and the tools we develop for our staff, across the globe, help them be more efficient in their daily working lives. This is possible due to the state-of-the-art data infrastructure our teams build and maintain. We work on critical national infrastructure that powers trade at the UK border and innovative data projects looking at global supply chains and future ways of trading. We believe in creativity, curiosity, diversity of thought, entrepreneurship, delivery focus and collaboration. We embody the DIT values to be expert, enterprising, engaged and inclusive. These values have helped us become a high performing team that continuously delivers high quality outputs.
Job description
As a Senior Data Engineer, you will design, implement, test and operate our data products, pipelines and services. You will work in a cross-functional, agile, team to integrate them into wider systems and business processes.
While you will have opportunities to contribute to the wider solution to satisfy our user needs, the team will rely on your ability to lead development of data products, to optimise the code, and help with database management. You will ensure internal and industry standards are followed in delivery of reusable solutions. You will help more junior team members to gain your level of expertise, and ensure the teams are working to the standards set by the lead data engineer. You will be confident in using languages like SQL and Python, and have deeper understanding of how they work.
Main responsibilities
You will be:
- Lead the design, build and test of data products and services using standard techniques and tools.
- Lead delivery of data solutions which are scalable, resilient and future proof.
- Have thorough understanding of common problems in databases, data processes, data products and services. You will understand and be able to apply typical solutions to those problems.
- Follow internal and industry standards to design, code, test and document complex data solutions.
- Undertake data profiling and source system analysis including techniques like reverse engineering.
- Be a key contributor in an Agile and cross-functional team
Person specification
- Agile methodologies. You can lead work following agile methodologies on data projects.
- Data development process. You can lead design, build and test of data products based on feeds from multiple systems using a range of different storage technologies and/or access methods. You create repeatable, reusable products.
- Data integration design. You know how to select and implement the appropriate technologies to deliver resilient, scalable and future-proofed data solutions.
- Data modelling. You understand the concepts and principles of data modelling and can produce relevant data models across multiple subject areas. You know how to reverse-engineer data models from a live system. You understand industry-recognised data modelling patterns and standards and when to apply them. You can compare and align different data models.
- Problem resolution (data). You know how to respond to problems in databases, data processes, data products and services as they occur. You can initiate actions, monitor services and identify trends to resolve problems. You can determine the appropriate remedy and assist with implementation of it as well as preventative measures.
- Programming and build (data engineering). You know how to use agreed standards and tools to design, code, test, correct and document moderate-to-complex programs and scripts from agreed specifications and subsequent iterations. You can collaborate with others to review specifications where appropriate.
- Technical understanding (data engineering). You understand core technical concepts like data normalisation, modelling, performance of data intensive solutions. You understand basic software development concepts.
It is desirable that you have:
- Communication skills (data). You know about the need to translate technical concepts into non-technical language and understand what communication is required for internal and external stakeholders.
- Cloud infrastructure. Experience with cloud infrastructure, especially AWS, and infrastructure as code tools, such as Terraform.
