Custom Tasks

• Build Website
• Build Scheduled Tasks
• Build Databases
• Build Reports

Azure DevOps Server 2019 Agent and Task changes

After upgrade to Azure DevOps Server 2019, you need to

• Upgrade your agents with newer version
• Modify definitions to use newer version of some tasks
  • Copy Files
  • Run PowerShell
  • Maven
    • Maven task now runs Maven directly instead of executing it from a PowerShell. Because of this, I had to change some path for Maven to find files.
    • ${project.basedir} = the root directory of your project.
    • ${project.build.directory} = ${project.basedir}/target

Build A Custom vNext Build Task

It really comes handy when you can build your own build tasks. Either for encapsulation, re-usability, or just to look clean in your build definitions (or even as part of Low-Code movement).

You can find some references from Microsoft here.  But that’s for VSTS. Here I am going to provide a simple example for TFS on-premises.

First, you need some TFS build modules. Download them from GitHub’s Microsoft/vsts-task-lib

Second, need the command line tool for uploading the package to your TFS server. You can download and install from

npm i -g tfx-cli

Third, create the actual build script file, the one you want this custom task to execute during the build. You can specify parameters which will be passed through the task interface.

[CmdletBinding()]param()
Trace-VstsEnteringInvocation $MyInvocationtry {     Import-VstsLocStrings “$PSScriptRoot\Task.json” Import-Module -Name “$PSScriptRoot\ps_modules\Invoke-MsBuild\Invoke-MsBuild.psm1” Import-Module -Name “$PSScriptRoot\ps_modules\MSBuildHelpers\MSBuildHelpers.psm1”
[string] $ProjectFile = Get-VstsInput -Name ProjectFile -Require    [string] $OutputPath = Get-VstsInput -Name OutputPath -Require    [string] $MSBuildParameters = Get-VstsInput -Name MSBuildParameters    [string] $UseDataScripts = Get-VstsInput -Name MSBuildParameters Write-Host “ProjectFile:***$ProjectFile***”    Write-Host “OutputPath:***$OutputPath***”    Write-Host “MSBuildParameters:***$MSBuildParameters***” $MSBuildParameters = $MSBuildParameters + ” /p:OutputPath=$OutputPath” $SourceDir = (Get-Item $ProjectFile).Directory.FullName Write-Host “SourceDir:***$SourceDir***” ### Build Project file ### if ((Invoke-MsBuild -Path “$ProjectFile” -MsBuildParameters “$MsBuildParameters” -ShowBuildOutputInCurrentWindow).BuildSucceeded -ne $true) { Write-Error “*** ERROR – Build did NOT complete successfully! – $ProjectFile ***” } else { Write-Warning “*** Build completed successfully! – $ProjectFile ***” } ### Copy hand-picked predeploy scripts to BuildOutput folder based on PreDeploy.txt $ScriptDirName = “Scripts\Pre-Deploy” $PreDeployFilename = “$SourceDir\$ScriptDirName\Pre-Deploy.txt” Write-Output “PreDeployFilename:***$PreDeployFilename***” [System.IO.File]::ReadLines($PreDeployFilename) | %{$src=”$SourceDir\$ScriptDirName\$_”; $des=”$OutputPath\$ScriptDirName\$_”; New-Item “$des” -ItemType File -Force; Write-Output “Copy $src”; Copy-Item “$src” “$des” -Force -Recurse} | Out-String | Write-Warning
### Copy hand-picked predeploy scripts to BuildOutput folder based on PostDeploy.txt $ScriptDirName = “Scripts\Post-Deploy” $PostDeployFilename = “$SourceDir\$ScriptDirName\Post-Deploy.txt” Write-Output “PostDeployFilename:***$PostDeployFilename***” [System.IO.File]::ReadLines($PostDeployFilename) | %{$src=”$SourceDir\$ScriptDirName\$_”; $des=”$OutputPath\$ScriptDirName\$_”; New-Item “$des” -ItemType File -Force; Write-Output “Copy $src”; Copy-Item “$src” “$des” -Force -Recurse} | Out-String | Write-Warning      } finally {    Trace-VstsLeavingInvocation $MyInvocation }

 

Forth, create a task.json file to define the new custom task.

{
“id”: “XXXXXXXX-XXXX-XXXX-XXXX-XXXXXXXXXXXX”,
“name”: “BuildDatabase”,
“friendlyName”: “Build Database”,
“description”: “Build Database”,
“author”: “My Name”,
“helpMarkDown”: “Build Database”,
“category”: “Build”,
“visibility”: [
“Build”
],
“demands”: [],
“version”: {
“Major”: “0”,
“Minor”: “1”,
“Patch”: “1”
},
“minimumAgentVersion”: “1.95.0”,
“instanceNameFormat”: “Build Database $(message)”,
“inputs”: [
{
“name”: “ProjectFile”,
“type”: “filePath”,
“label”: “Project File”,
“defaultValue”: “”,
“required”: true,
“helpMarkDown”: “Relative path from repo root of the project file.”
},
{
“name”: “OutputPath”,
“type”: “string”,
“label”: “Output Path”,
“defaultValue”: “”,
“required”: true,
“helpMarkDown”: “Path of the Output directory.”
},
{
“name”: “MSBuildParameters”,
“type”: “string”,
“label”: “MSBuild Arguments”,
“defaultValue”: “”,
“required”: false,
“helpMarkDown”: “Additional arguments passed to MSBuild.”
},
{
“name”: “DeployScripts”,
“type”: “string”,
“label”: “Include Pre/Post-Deploy Scripts”,
“defaultValue”: “false”,
“required”: false,
“helpMarkDown”: “If to run pre/post deploy scripts manually without using build action.”
}
],
“execution”: {
“PowerShell3”: {
“target”: “BuildDatabase.ps1”
}
}
}

Fifth (optional), prepare a icon file for showing up on the Build Task list.

Sixth, upload to the server

tfx build tasks upload –task-path .\BuildDatabase –auth-type basic –username <domain\username> –password <password> –service-url http://<tfsserver&gt;:8080/tfs

That’s it. You will then see your custom tasks showing up on the available task list when you try to add build step.

And you can see how the parameters in your build scripts look like at the front end.

A Journey of Implementing TFS 2015

These were teams with basic Agile (Scrum & Kanban) training and been using TFS only as source code repository in the past. Developers had been specialized in each of their own applications and were about to start pooling together to form a more collaborate force. So no branching (and merging), no work items, no work item association at check-in, and of course no standard build and deploy processes and environments yet. What a exciting journey this will be!

“The main thing is to keep the main thing the main thing”      ~ Stephen Covey

I want to keep the interest of early adopters by keeping the system stable and up-to-date. In the meantime, seeing is beleiving. It helps to have environments for me to demo and POC some ideas for my potential users. Then it will come down to prioritize what goes first with limited resources and time. Few important things for me and the teams are

• A healthy and up-to-date TFS environment
• Provide custom solutions which also aleign with industry trends and new technology
• Evangelize and POC for potential proposal

However, business priority overrules technology came to the equation when trying to prioritize the implementation tasks. Furthermore, those priorities changed dynamically as well. This challenge is pretty common in many companies for IT department. The only difference in my case is that now the IT teams are the end users for me while I am also part of IT too.

So we did the upgrade from TFS 2013 to TFS 2015 after applying few updates for TFS 2013. The goal is to keep the system current. Interestingly, the side benefit of the multiple smooth upgrades also helped me gain some trust gradually. There were some big database schema changes between 2015 and 2013 followed by additional changes in TFS 2015 Update 1 & 2. Application layer wise, Release Management is fully integrated into TWA (TFS Web Access) now, TFS extension market place is available, and more supported custom widgets are possible starting in Update 3 now.

To gain more buy-in, I explorerd and eventually customize two version of Work Item card printing solution for some teams using physical boards on the walls with sticky notes. The organizational culture is not too exciting about new technology and changes. So for those teams finally took the leap of faith by jumping on new Agile practice and set up boards on the wall few months ago, printing out cards is a most acceptable step because “moving cards” was what they were trained for Agile. I did get sidetracked on some customization and support for this printing effort. But during the processes, I also got to know more teams’ unique requirements and the chance to show-and-tell some TFS features.

As more and more data moved into TFS from various traditional tracking means i.e., sticky notes, Excel spreadsheets, SharePoint lists, etc. along with improved and more friendly TWA, department head finally agreed to go electronic and move all boards to TFS if we can solve some of his concerns and requirements.

• Can TFS report departmental work loads, trends, unexpected activitiesetc. to help explaining the delivery to business users
• Each team has their own unique flows of running the boards. Would it be too much customization for future maintenance. Some of the teams suffered from painful upgrades caused by customization in the past and very sensitive about it.
• Can TFS boards be flexible enough to fit different teams special needs?
• Can his teams do this without affecting other more established teams?

Fortunately, we moved on to get everyone on TFS boards for thess multidiscipline teams.

Priority shifted in the middle of the board migration project. Some Dev team members got excited about what TFS can do and promised business users that TFS build service can solve their worsen code delivery issues caused by human mistakes. While TFS does provide the integration and automation, there are definitely more factors to consider when implementing an automated CI.

Branching & Merging

This is not a huge development team but most of the members do not have experience with branching and merging. So we start from scratch to gather current processes, pain points, workspace setup, and needs and wants.

Standardize build processes

From setting up build service (I also noted more details about how to decide the right build service) and cleanup some library referencing errors, standardize output files to manage config files.

Release Management

Not only the development team was doing the deployment in Production, The developer who changed the code is actually the one deploy the code to production. Because of this, deployment approval has to be carefully managed.

Intelligence

Prevent human errors like partial check-in, manual steps, and inconsist deployment flows were the original reasons for this initiative. So we do the work item association at check-in, label them by external ticket ID which is the system for Change Management so both developers and business users can clearly see the details for the delivery from TWA widgets.

Training

It is so great to partner with a reputable PM when rolling out TFS boards. We are basically doing both Agile process refreshment and TFS training at the same time. In addition, we also collect valuable user feedback during the roll-out training for both board Admin and users.

Migration

For some of the teams have been using other computer systems like Excel, SharePoint and other 3rd-party ticketing systems, it is just making more sense to help them migrating their records into TFS.

Reporting

Now that data are all set to go into TFS, we are ready to process them and get it out in a more meaningful ways. As far as tools go, there are some options being discussed.

• SSRS reports
• Power View
• Power BI

Integration

Priority shifted again here as one might expect. Management is thinking about using different ticketing system for Change Management. Although it is still in its early stage, our leader wants us to prepare the integration between TFS and the new CM system. The goal is reduce the redundent data entry overhead between systems. I decide to use Web Hook as the service hook on the TFS side and is currently exploring to subscribe the other side’s web service.

Use TFS 2015 new integrated Release Management with vNext builds for Continuous Integration (CI)

Choises of Build Definition:

• xaml based
  • WPF framework
  • Customize activity with custom DLLs
• vNext
  • Support multiple platforms
  • Local Configuration Management (LCM)’s DSC
  • Agentless

Choices of Release Management:

• Stanalone Release Management (InRelease)
  • Standalone databases are gone during the recent updates. You will no longer see the ReleaseManagement database on the TFS DB server
  • Standalone executables are going away pretty soon. Microsoft has stopped providing updates and will just fade it out in the next few releases
• Integrated with TFS Web Access (TWA)
  • Web-based interface
  • Browser friendly
  • Integrated with TFS databases
  • Custom tasks/steps available (extension)
  • Environment-oriented deploy steps
  • Support multiple platforms

Customization & Integration

• Build Tasks
  • Cleaner look, if you have multiple steps, you may want to combine them together into a custom build task. Plus you can have your own icon.
• TWA Hubs
  • A developer hack for on premises TFS for now. But works
• TWA Widgets
  • A quick view especially useful for business users
  • 4×4 limitation
• Service Hooks
  • Integrate TFS with other ticketing systems using Restful + json
• Reports
  • SSRS
  • Excel + Tabular mode
  • Power BI