Senior Data Engineer
Job Title: Senior Data Engineer
Department: Knowledge Management
Location: Tampa, FL
Summary of Duties & Responsibilities
The BST Global AI & Analytics product team delivers data-driven business analytic solutions for the professional consulting market through artificial intelligence (AI), Big Data architecture and machine learning expertise. The Senior Data Engineer is a key member of the AI & Analytics team, providing domain expertise and leadership to deliver data-related products, features, and functionality that serve business and technical leaders.
The Senior Data Engineer must have a thorough knowledge of Microsoft Big Data solutions. This individual will be responsible for defining high-volume data processing pipelines in a multi-tenant environment, verifying data integrity and curating data for consumption by data warehouse, third-party data analytics tools and machine learning models. The ideal candidate is an exceptional critical thinker with strong business acumen, and is adept at solving complex problems and creating elegant business solutions. You will be involved in the design and development efforts for our Big Data solutions, including data lake, business intelligence solutions, machine learning, data pipeline and cloud-based data warehouse products.
- Work with clients to understand data analytics requirements; analyze data to ensure it meets specific data management, governance and quality assurance requirements before processing within the data lake and data warehouse
- Work with product managers, database architect or BI architect to understand data pipeline and data life cycle processing requirements and patterns
- Install and configure data sources for use by the data pipelines
- Work with the database architect to define data management, governance and quality assurance requirements to manage the entire data processing life cycle
- Act as a key contributor to all phases in the design and development life cycle of analytic applications utilizing Microsoft Azure and BI technology platforms
- Curate data for analyses, business reviews and operational data analysis demands
- Use an agile approach to define requirements, design and develop data pipeline solutions to enable near real-time change data capture (CDC) and data consumption
- Apply data management, governance, and quality processes and tools to correlate disparate sources
- Apply strong analytical skills to translate complex business requirements into sound data management and data governance solutions
Skills & Competencies
- Experience with Azure products and services, including Azure Data Lake Gen2, Azure Data Bricks, Azure SQL Server and Azure Analysis Services
- Advanced SQL development skills, broad exposure to all language constructs
- Experience with Microsoft on-premise SQL Server (2017 or higher) and Azure SQL Server technologies, including broad experience with SQL Server capabilities and tools (e.g., CDC, Columnstore Index, In-Memory Table, SSAS Tabular, DAX, T-SQL, SSIS)
- Experience in Power BI, Power BI Embedded, Power BI Services, Power BI Gateway and Power BI Dataflow
- Experience using Azure DevOps and CI/CD as well as Agile tools and processes including Git, Jenkins, Jira and Confluence
- Experience with data integration through APIs, Web Services, SOAP and/or REST services
- Experience with OLTP data modeling, dimensional data modeling, composite modeling, data transformation and designing the most optimal analytical data structures for near real-time data analytics
- Additional programming experience is a plus (preferably.NET) or other languages such as Python, Scala, R.
Education or Prior Work Experience
- Bachelor’s degree in Computer Science
- 10+ years of experience designing and developing complex data analytics solutions
- 5+ years of experience with Microsoft Big Data solutions
Manager, Software Engineering
Up to 15%
This job operates in a professional office environment. This role routinely uses standard office equipment such as computers, phones, photocopiers, filing cabinets and fax machines.
While performing the duties of this job, the employee is regularly required to talk or hear. The employee frequently is required to stand; walk; use hands to finger, handle or feel; and reach with hands and arms.
BST Global is the leading provider of enterprise business management software and service solutions for architecture, engineering, and environmental Professional Service Organizations globally. Today, over 100,000 end users across six continents and 50 countries rely on BST Global’s software solutions to manage their projects, resources, finances and client relationships.
Founded in 1971, the privately held BST Global is headquartered in Tampa, Florida, USA and is a global organization with a presence in the Americas, Europe, Africa, Asia and Australia. With over forty years of experience, BST Global is the industry leader and is strategically positioned for growth. Our employees enjoy a generous benefits package including paid time off, medical, dental, disability, life insurance, and 401k.