This blog describes the process required to set up and configure Continuous Integration (CI) with Dynamics 365 for Operations, Lifecycle Services, and Azure DevOps (previously known as Visual Studio Team Services).
- VSTS optional build agent configuration can be performed. See Appendix A.
- VSTS project setup and basic folder configuration for D365 projects must be performed. See Appendix B.
Terms and Definitions
Throughout this blog, specific terms are used to refer to different aspects of the Dynamics 365 for Operations system. At the end of the blog an abbreviations addendum is provided to clarify various terms and acronyms used throughout the blog.
Regardless if you use a shared account (such as AXAdmin) or an organization account, the account performing the tasks must have full administrator rights on both the LCS project and the VSTS project collection. It is recommended to use a shared account (such as AXAdmin) as the primary administrator account for the D365 instance, but all these activities can be performed with any user account in the organization.
- Project Owner permissions on the LCS “Implementation Project”.
- Project Collection Administrator permissions on the VSTS site (global).
All tools required for this task are included with Windows, no additional software install is required.
- A web browser is required for LCS and VSTS. Edge or Chrome* are recommended.
- Microsoft Remote Desktop Client is required to connect to the build environment.
NOTE: *Chrome is not included with Windows. In some scenarios Internet Explorer will be present as the default browser instead of Edge.
- A Microsoft Developer Network (MSDN) subscription (Pro or Enterprise) is required for the user or service account that performs the configuration and setup work.
Connect Implementation Project to VSTS
- Log in to LCS and open the “Implementation Project” for the customer.
- Open “Setup Visual Studio Team Services” by clicking on the available links.
- Click “Setup Visual Studio Team Services” button.
2. In a separate browser window/tab, log in to VSTS.
3. Click on your name in the upper right corner and select “Security” from the menu.
NOTE: Your user interface may appear different than what is shown in the screenshots, depending on your preview features settings. The same instructions will work in both navigation schema, although the menus may appear slightly different.
4. Click the “Add” button. (In Preview Navigation, this will be “+ New Token”)
5. Create a token.
6. Copy token key to LCS.
7. Enter Visual Studio Team Services site URL.
NOTE: You cannot use the "new" Azure DevOps style URL for the project, you must use the older style <hostname>.visualstudio.com URL in LCS.
8. Click “Continue” button.
- Select the VSTS Project (configuration required, see Appendix B.)
- Review settings, click “Save” button.
- Once the configuration is complete, scroll down and click the “Authorize” link to finalize the connection.
2. Click the “Accept” button.
Configure Build Triggers for CI
After completing the previous process, and deploying your system, a Build Definition will be created automatically. By default, a Continuous Integration and a Gated build trigger are created. Adding a scheduled build is simple.
NOTE: You can manually activate a build from the Builds page by clicking the "Run" button at any time.
- Log in to VSTS.
- Select Pipelines -> Builds page from navigation.
- Select “Unified Operations platform – Build Dev” and click Edit.
4. Select the Triggers pivot tab and click “+ Add” button.
5. Define your scheduled build timing. By default, the form will have weekdays selected. You can add weekend days here, or create multiple scheduled builds with your desired timing.
Configure Batch Queueing for CI
By default, continuous integration is configured to attempt to build after every code check-in. If your development team is checking in code regularly, this can cause conflicts in the usage of the build agent. If you enable continuous integration, you should also enable batching.
Following the steps from the previous process, navigate to the Triggers pivot tab on the Builds page once more, and highlight the Continuous integration build trigger.
- Check “Enable continuous integration”
2. Check “Batch changes while a build is in progress”
Configure Deployment Settings (VSTS Requirements)
- Log in to LCS and open the “Implementation Project” for the customer.
- Click the “Configure” button on your Sandbox Build environment (name may vary).
3. Select application/platform version and topology. Accept the default (latest) version and select the Build and Test topology.
4. Click “Advanced settings” button.
5. On the Visual Studio Team Services page, fill in values for Build Agent Name, Build Agent Pool, and Branch Name as required.
NOTE: There are additional configuration pages that need to be configured before deployment. These settings are outside the scope of this blog. Please consult with your project manager or architect for the required settings for other pages.
Optional Build Agent Configuration
As part of the LCS deployment of a Microsoft-hosted D365 environment , you can configure a Build Agent for creating packages based on a schedule or check-in.
This agent runs as a service on the D365 BUILD instance, and you can customize the Agent Pool and Agent Name, as well as the Branch in VSTS that is being used for package creation. This configuration cannot be changed after the environment is deployed, except by deleting and redeploying the environment again.
We recommend that the Agent Name be customized to represent the environment in which the Agent is installed, as this name will be shown in the VSTS Agent Pool definitions.
Create Custom Agent Pool
- Log in to VSTS and select your Project.
- Click “Project settings” and then select the “Agent pools” page.
- Click “New agent pool…”
4. Enter a new pool name and click OK.
NOTE: This will create an agent pool at the Organization level. If you need to permanently delete the Agent Pool you will need to delete it at the Organization level, not the Project level. Click the "Manage organization agent pools" link on the Agent Pools page.
Configure Azure DevOps Repository for D365
- Create Trunk, Branch, Metadata and Projects folders as shown here. By default, the Branch name is Main, your branch name may vary.
2. Log in to the BUILD VM using Remote Desktop Connection client. (You may obtain the RDP file and credentials from the environment details page in LCS)
3. Open Visual Studio and perform initial configuration. (Log in to Visual Studio and accept license)
4. Select “Team -> Manage Connections…” from the menu.
5. Select the appropriate existing TFS server or click the “Servers…” button to configure a new connection if required.
6. Configure your workspace mappings for the Metadata (package) folder. In Source Control Explorer, select the Metadata folder, click on the “Not mapped” link and then browse to the “Service volume” and select the AosService/PackagesLocalDirectory folder.
NOTE: Some topology deployments have different names for the packages folder and different storage layouts. Your Service volume may be a different drive letter and your package folder may have a different name.
7. Configure your workspace mappings for the Projects folder. In Source Control Explorer select the Projects folder, click on the “Not mapped” link, and then select your project file repository folder.
Additional CI/CD Build Automation Details
CI/CD Build Steps
I. Get Sources – Uses a Server path mapping to define the branch, as specified in the Deployment settings: $/<Project name>/Trunk/<Branch>. This reflects the folder structure created previously during VSTS setup.
A. Prepare for build – A DynamicsSDK PowerShell script that prepares the database for a build.
B. Set Model Versions – A DynamicsSDK PowerShell script that sets up model build version details.
C. Build the solution – Executes MSBuild to create the output binary packages and logs for the build.
D. Database sync – Executes MSBuild to synchronize changes in the D365 database. This step will roll back to the checkpoint database created in step “A.” in case of failure.
E. Deploy Reports – A DynamicsSDK PowerShell script that deploys SSRS report definitions to the SQL Server Reporting Services database.
F. Generate Packages – A DynamicsSDK PowerShell script that creates the package files for the Asset Library.
G. Copy Files to: Staging Directory – A build step which copies the output packages to a staging directory in preparation for publish.
H. Publish Artifact: Packages – A build step which pushes the staged packages to the Asset Library.
I. Test Setup – A DynamicsSDK PowerShell script that prepares test projects to be executed against the latest build.
J. Execute Tests – Using Visual Studio Test, test projects are run against the latest build.
K. Test End – A DynamicsSDK PowerShell script that collects the test results.
L. Copy Files to: Staging Directory – A DynamicsSDK PowerShell script that copies the output packages to the staging directory in preparation for publish.
M. Publish Artifact: AdditionalLogs – A build step which pushes the test results to the Asset Library.
Additional Manual Scheduling
Continuous Integration versus Gated check-in
The main difference between Continuous Integration and Gated Check-in is the additional validation step performed on the code package before build and publish steps are taken. With Continuous Integration, by default, any code check in will create a build. If you have many developers working in this manner, it is a good idea to enable Batching of these builds so they do not conflict with each other and will instead run in sequence.
When using a Gated Check-in, the contents of any given commit to the repo will be first built as a shelf set and validated for success before then beginning the normal Continuous Integration build. This provides extra protection for large development teams to ensure that only properly validated, functioning and successfully built code will be introduced into the environment. The downside is that the overall build process becomes much longer.
Multiple Scheduled Builds
You may add as many Scheduled build as you would like. By default, the schedule will be at Midnight UTC on weekdays. It typically makes sense to begin a scheduled build some time after the end of the day for developers. This could vary depending on the time zones and typical work patterns of the team, so it might make sense to have multiple schedules for multiple time zones.
Throughout the blog we use specific terms to refer to different aspects of the Dynamics 365 for Operations system.
Asset Library – A location in LCS where all project assets such as data, code and ISV packages are staged for deployment.
Azure – Microsoft’s cloud-based hosting service.
Azure DevOps – The service formerly known as Visual Studio Team Services.
Business Process Library – Business logic and testing workflows that can be executed from LCS.
Common Data Model – Microsoft’s implementation of OpenAPI, OData and other standard data entity interfaces for RESTful services.
Data Entity – A standard OData schema object available in Microsoft’s Common Data Model.
Data Package – A zip file containing data and metadata for one or more entities.
Document Deliverable – A Functional or Technical design that is the core of all work items, connecting business requirements to development requirements.
Environment – A single copy of Dynamics 365 that is used for a specific purpose such as Standard Acceptance Testing, Build and Test, Dev and Test, or Production. Environments can be of several tiers which defines the topology of the environment.
Instance – Can be used interchangeably with Environment.
Lifecycle Services – A service provided by Microsoft to manage Dynamics 365 implementations.
LogicApps – An OpenAPI service available in Azure.
Model – A package of code customizations for an implementation project.
Microservice – A modern service implementation practice that isolates minimal functionality into discreet service units.
Package – A deployment unit of custom code extensions or data that can be applied to an environment through LCS.
PowerApps – A service available in Azure that provides rapid development of simple apps based on the Common Data Model.
PowerBI – A service available in Azure that provides insights about data in Dynamics 365, and to other services connected through the Common Data Model, such as PowerApps.
Sandbox – An environment for testing and prototyping solutions. All non-production environments are considered sandbox in D365.
Service Endpoints – The connection points for various services, including ports, schema (WSDL/Swagger), URIs and URLs for various components.
Subscription Estimator – A process and tool in LCS that must be performed prior to the deployment of Production. Transaction and user volumes are estimated, and Microsoft uses this information to size the Production environment. This step is required and cannot be skipped in any way.
Visual Studio Team Services – Microsoft’s team software development platform, based on Team Foundation Server, hosted in the cloud as a SaaS solution.
AOS – Application Object Server
AOT – Application Object Tree
API – Application Programming Interface
BI – Business Intelligence
CAR – Customization Analysis Report
CD – Continuous Delivery
CI – Continuous Integration
CIL – Common Intermediate language
D365 – Dynamics 365 for Operations (variations)
DSL – Definitive Software Library
IaaS – Infrastructure as a Service
KPI – Key Performance Indicator
LCS – Lifecycle Services
OAuth – Open Authentication Protocol
OData – Open Data Protocol
PaaS – Platform as a Service
RDP – Remote Desktop Protocol
REST – Representational State Transfer
SaaS – Software as a Service
SAT – Standard Acceptance Testing
SSAS – SQL Server Analysis Services
SSMS – SQL Server Management Studio
SSRS – SQL Server Reporting Services
TFS – Team Foundation Server
URI – Uniform Resource Identifier
URL – Uniform Resource Locator
VS – Visual Studio
VSTS – Visual Studio Team Services
WCF – Windows Communication Foundation
WPF – Windows Presentation Foundation
XAML – XML Application Markup Language
XML – Extensible Markup Language
XPO – X++ Object File