Software Developer - ETL - Senior
Senior onsite ETL Developer to design, develop, maintain, test, and document database backend and ETL applications for Data Warehouse and Business Intelligence solutions using Azure, AWS, Databricks, SQL, and related technologies.
- Organization
- Central Agencies Cluster
- Location
- Toronto, Ontario
- Work arrangement
- On-site
- Employment type
- Contract
- Compensation
- $723/day
- Reference
- RQ11713
Overview
Senior ETL Developer assignment with Central Agencies Cluster, Ministry of Treasury Board Secretariat. The consultant will design, develop, and maintain database backend and ETL applications for Data Warehouse and Business Intelligence solutions. The successful candidate will use ETL languages and development techniques to build and expand application functionality by writing high quality code using appropriate patterns, ensuring applications remain scalable while complying to standards and following best practices. The consultant will test and debug applications, ensure software is properly documented, collaborate professionally as part of a development team, and communicate with clients.
Onsite Work Requirement: The successful candidate is expected to work onsite five days per week, for 7.25 hours per calendar day, between the standard working hours of 8:00 AM to 5:00 PM (excluding lunch breaks), Monday to Friday inclusive, at the identified OPS office location.
Responsibilities
- Design, implement, and continuously expand data pipelines by performing extraction, transformation, and loading activities.
- Translate technical systems specifications into working, tested applications.
- Develop detailed programming specifications, write and/or generate code, compile data-driven programs, maintain applications, and conduct unit tests.
- Resolve and troubleshoot technical problems arising during the use and operation of software packages, including providing technical assistance in implementation, conversion, and migrations.
- Design, develop, and maintain database backend and ETL applications for Data Warehouse and Business Intelligence solutions.
- Use knowledge of ETL languages and development techniques to build and expand application functionality by writing high quality code using appropriate patterns.
- Ensure applications remain scalable while complying to standards and following best practices.
- Test and debug applications and ensure that software is properly documented.
- Collaborate clearly and professionally as part of a development team and communicate with clients.
- Investigate data to identify potential issues within ETL pipelines, notify end-users, and propose adequate solutions.
- Prepare detailed documentation for further reference.
Mandatory Qualifications
- Experience with the Microsoft suite of technology including: Azure Data Factory, Azure SQL Database, Azure Data Lake, and Power BI.
- Experience with Databricks Platform.
- Experience with the Amazon Web Services (AWS) suite of technology including: Amazon Aurora, Amazon Relational Database Search, Glue, OpenSearch, etc.
- Experience with developing data extraction, transformation, and load programs (daily and initial load) functionality on a wide range of data repositories (structured and semi-structured files, relational and multi-dimensional data stores).
- Experience with developing, implementing, and maintaining schedule/dependency logic for Extract Transform Load (ETL) scripts.
- Experience with data integration, data cleansing, and data analytics.
- Experience with data modeling and design principles for data marts and data warehouses.
- Experience in database management and administration.
- Experience in constructing complex SQL queries and performance tuning.
- Experience with Git and knowledge of source control strategies.
- Experience creating technical documentation including ETL source-to-target mappings, data model diagrams, architecture artifacts, detailed design documents, etc.
- Experience in eliciting, documenting, and validating business requirements.
- Experience in software development lifecycle from Development, to Testing and Quality Assurance, to Delivery.
- Solid communications skills: the ability to understand, comprehend, and follow instructions and requirements to communicate clearly and effectively both verbally and in writing.
Preferred Qualifications
- Knowledge and understanding of Accessibility for Ontarians with Disability Act (AODA) and related regulations and standards.
- Experience designing visual dashboards to highlight trends and Key Performance Indicators (KPIs), and developing canned reports (scheduled and ad-hoc).
- Experience with programming languages such as C#, developing application code such as APIs, Functions, etc.
- Experience writing scripts with PowerShell and other scripting languages.
- Knowledge and experience with programming Internet-ready applications.
- Knowledge and experience in rapid application development (RAD) methodologies.
- Knowledge and understanding of Information Management principles, concepts, policies and practices.
- Experience with middleware and gateways.
- Experience reviewing, analyzing, and modifying product installation scripts including encoding, testing, debugging.
- Ability to provide post-implementation support and resolve any post-implementation technical issues.
- Experience conducting design walkthrough sessions with project team.
- Ability to provide user and system documentation as required.
Skills and Experience
- Experience in programming and analysis; specialized software package support at the specified experience level.
- Ability to collaborate with IT Professionals throughout the Software Development Life Cycle.
- Experience in structured methodologies for the development, design, implementation and maintenance of applications.
- Experience in design, code, test, debug and document applications.
- Experience in the use of object and/or third generation language development tools.
- Experience in one or more programming languages.
- Experience in application design, latest design patterns, deployment and troubleshooting.
- Experience with relational and hierarchical database technologies.
- Experience in the use of information retrieval packages using query languages.
- Experience with one or more communications protocols.
- Experience eliciting and documenting information from diverse business area stakeholders and subject matter experts.
- Excellent analytical, problem-solving and decision-making skills; verbal and written communication skills; interpersonal and negotiation skills.
- A team player with a track record for meeting deadlines.
- Knowledge and experience in gathering requirements and business process knowledge in order to transform the data in a way that is geared towards the needs of end users.
- Knowledge and experience maintaining and improving already existing processes.
- Knowledge and experience ensuring that the data architecture is scalable and maintainable.
- Knowledge and experience working with the business in designing and delivering correct, high quality data.