A simple task to view the Pipeline Agent File System

Introduction

There are several predefined variables that you can use within an Azure DevOps pipeline, some of which allow you to reference locations on the pipeline agent file system. Examples of when to use them might be to pick up a file from the downloaded repository or to describe where to output a build. For example the variable $(Build.ArtifactStagingDirectory) will be replaced during the defining of the pipeline job with the respective filepath.

I find it difficult at times to work abstractly with the file system, and so I use this template to visualise the file system for debugging purposes, to observe the outcome of my code, and to make sure I am picking the right variable.

Predefined variables

Predefined variables are automatically set by the system and are mostly read-only. In YAML pipelines, the variables can be referenced as environment variables.

I tend to forget the filepath locations and I find it hard to work abstractly with a filesystem on a remote machine that I cannot see! I have written a simple task that displays the output of some of the locations I most commonly use and then displays the entire contents of the file system. I can turn this step on whilst developing so that I can evaluate the file system and determine whether my files are located in the desired location. It also helps me work out the right variables to use, and to triage issues.

My file system snippet

This is the snippet I use to view the file system on the agent:

- script: |
    echo ##[debug]List of built in variables
    echo Agent.WorkFolder: $(Agent.WorkFolder)
    echo Build.ArtifactStagingDirectory: $(Build.ArtifactStagingDirectory)
    echo Build.BinariesDirectory: $(Build.BinariesDirectory)
    echo Build.SourcesDirectory: $(Build.SourcesDirectory)
    echo ##[debug]Showing Agent.WorkFolder'
    tree $(Agent.WorkFolder)/1 /f /a    
  displayName: Display tree of $(Agent.WorkFolder)
  enable: true

You can copy and paste this directly into a pipeline job and it will run seamlessly.

Note the enable: true switch at the end, by setting this to false, the step will be skipped. Useful if you want to retain the snippet and turn it on when required in the future. When the step runs it will print the location of four predefined variables, Agent.WorkFolder, Build.ArtifactStagingDirectory, Build.BinariesDirectory and Build.SourcesDirectory. It will then visualise the file system using the tree command.

Note that the tree command accepts a path, which I have set to Pipeline.Workspace the /f switch specifies that files should be displayed as well as directories, the /a switch specifies to use text characters instead of graphic characters to show the lines representing subdirectories, something that needs to be turned on for the output from pipelines.

Here is the output from one of my pipeline runs:

##[debug]List of built in variables
Agent.WorkFolder: D:\a\1
Build.ArtifactStagingDirectory: D:\a\1\a
Build.BinariesDirectory: D:\a\1\b
Build.SourcesDirectory: D:\a\1\s
##[debug]Showing Agent.WorkFolder'
D:\A\1
+---a
+---b
+---s
|   |   .gitignore
|   |   README.md
|   |   
|   +---.ado
|   |   +---devops
|   |   |       debug.yml
|   |   |         
|   |   +---powerbi
|   |   |   |   check.yml
|   |   |   |   publish.yml
|   |   |   |   
|   |   |   +---stages
|   |   |           deploy.yml
|   |   |           test.yml
|   |               
|   +---.github
|   |   \---workflows
|   |           with-federated-credential.yml
|   |           with-secret.yml
|   |           
|   +---.scripts
|   |       test-api.ps1
|   |       
|   +---.tabular
|   |       license-FastWildcardMatching.txt
|   |       license-TabularEditor.txt
|   |       license-TreeViewAdv.txt
|   |       Microsoft.AnalysisServices.Core.dll
|   |       Microsoft.AnalysisServices.SPClient.Interfaces.dll
|   |       Microsoft.AnalysisServices.Tabular.dll
|   |       Microsoft.AnalysisServices.Tabular.Json.dll
|   |       Microsoft.Identity.Client.dll
|   |       Microsoft.IdentityModel.Abstractions.dll
|   |       Rules-Dataset.json
|   |       System.Buffers.dll
|   |       System.Diagnostics.DiagnosticSource.dll
|   |       System.IO.Compression.dll
|   |       System.Memory.dll
|   |       System.Net.Http.dll
|   |       System.Numerics.Vectors.dll
|   |       System.Runtime.CompilerServices.Unsafe.dll
|   |       TabularEditor-license.rtf
|   |       TabularEditor.exe
|   |       TabularEditor.exe.config
|   |       TabularEditor.zip
|   |       
|   +---docs
|       |   pull_request_template.md
|       |   
|       \---pull_request_template
|           |   changes_to_unit_tests.md
|           |   
|           \---branches
|                   develop.md
|               
\---TestResults

I typically place this within a template file because I might use it within several different pipelines, or at different times in the same pipeline to observe the changes in the file system. For simplicity, I am not going to walk through templates in this blog post.

Issues

The only issue that you might run into is too many results from the tree command. I have had this in the distant past and I cannot remember the limits, but should you encounter this issue you can either remove the f switch (so you only return the directories) or simply split the tree command into multiple paths.

Conclusion

This is a really useful snippet that I always have available within projects, it means that I can assess the remote file system during development and also double check that am using the correct predefined variables for my pipelines.

References

• Use predefined variables
• Artifacts in Azure Pipelines
• tree command