On October 20, 2022, the Healthcare Intelligence (HI) team released the next generation of our Continuous Integration and Continuous Delivery (CICD) pipeline tool, Sauron. HI currently uses Sauron to validate and deploy code changes by our engineers for data reporting and software application tools.
Sauron is intended to be CICD as a service, built for HI partners. It is on track to enable automation of all types of application releases into our modern data estate, like Databricks Notebooks, CDW changes, web applications, and other Azure-based development. HI is committed to contributing to the ongoing efforts to increase efficiency, operationalize our platforms, and deliver value. Sauron’s modular design integrates readily with many source code repositories and target platforms.
Who
The HI DevOps Engineering team developed Sauron for the Data Analytics, Data Engineering, and Data Science teams who use:
- Cloud Data Warehouse (CDW)—the data engineering and computing for production BI and analytics reporting platforms
- Databricks—the data science and research environments
- Control-M—data extract, transfer, and load (ETL) job scheduling system that moves Providence data between our systems and data platforms
What
Sauron is an internally developed Python application based in Azure using Azure Git Repos for source code repositories. It makes the automation and integration with Azure-based infrastructure and applications smooth and seamless.
With a modular design on both ends, many code repositories can be enrolled as sources, and many platforms can be attached as validation and deployment targets. This approach saves considerable time and eliminates the need to create a new pipeline for every project that depends or runs on Azure resources like CDW and Databricks. Sauron is integrated into Azure DevOps. Developers use it within the Azure DevOps web portal for a seamless validation and deployment experience with its pull request graphical user interface.
Features
- User-defined order of operations for code deployments
- Databricks notebook deployments including notebook validation
- Automatic promotion of Control-M jobs from development to production
- Reduction of validation runtime by 50%
- Weekly deployment report
What’s Next
Soon the Sauron pipeline will expand its offerings to include:
- Automated unit testing for CDW and Databricks
- Support for web app deployment
- Support for Power BI
- Support for other Azure based applications
More info
Please contact the HI DevOps Engineering Team for questions or assistance with enrolling your repositories into Sauron.