Senior Azure Synapse Data Engineer
UK (Fully Remote)
£92,000 - £115,000
Sponsorship cannot be provided
We are looking for a Senior Azure Synapse Data Engineer with exceptional technical, analytical, communication and project management skills to join our team of data and analytics experts, helping organize and leverage arguably the richest, broadest data available on work and people, and drive the build-out of a scalable, distributed data estate and supporting applications.
The Senior Azure Synapse Data Engineer will support our move to a modern data architecture and build-out of our data estate. Core activities include automating data ingestion from internal and external tools/platforms; identifying and linking individuals across different datasets/platforms for further analytics; creating sophisticated ETL processes to clean and organize our data using Azure Synapse pipelines, and providing Business Intelligence to establish the breadth, depth, and quality of the data.
Architecture design. Design, develop and maintain the architecture of the Azure data estate (data lake, SQL Analytics store, pipelines, ML/AI environment and network) to meet business requirements.
Data acquisition and integration: ensure feeds from various platforms - primarily internal but increasingly external are established. Use programming skills to develop, customize and manage ingestion tools, storage accounts, and analytical applications and their access.
Perform ETL processes for optimal extraction, transformation, and loading of data from a wide variety of data sources using Azure Synapse tools and technologies.
Data pipeline development/testing. Test the reliability and performance of each part of a system during the development phase. Identify, design, and implement process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability and data quality.
Track pipeline stability. Monitor the overall performance and stability of the system and ensure automated parts/scheduled tasks are monitored.
Manage structured and unstructured data and meta-data stored in the data estate. Ensure availability of clean, transformed data to meet business needs.
Machine learning algorithm deployment. Deploy Machine learning models (designed by data scientists) into production environments and help manage these environments.
Partner with IT to provide Business Intelligence tools/infrastructure. Set up tools, views, and access permissions for different user groups enabling them to view data, generate reports, and create visuals. Work with data scientists to strive for greater functionality and use of our data.
- Will likely have a degree in Computer Science, Statistics, Informatics, Information Systems or quantitative field.
- Microsoft Azure Synapse experience is essential.
- Excellent communication skills with the ability to work within a team, across the business, and build strong relationships.
- End-to-end Data Warehouse experience: ingestion, ETL, 'big data' pipelines, data architecture, message queuing, stream processing, BI/Reporting, and Data Security.
- Ability to build processes supporting data transformation, data structures, metadata, dependency, and workload management, as well as the ability to manipulate, process, and extract value from large, disconnected structured and unstructured datasets.
- Advanced SQL/relational database knowledge (including SSIS), query authoring (SQL).
- Knowledge of SQL Server Always Encrypted.