Configure Test Client Locally (On-Prem) with VS Team Services Release

To configure an on-premise test client machines with Visual Studio Team services release follow the below steps.

Diagram 01

1. Setup test client machine with WinRM  if not already installed. (WinRM is automatically installed with all currently-supported versions of the Windows operating system.)

2. Enable PowerShell remoting in the test client machine.

Enable-PSRemoting

3. Add test client machines in machines tab and create machine group. This is available in VSTS Test tab as of now.

Do not worry about the warning that it will be removed. It will just be moved under release tab will be a more enhanced feature. The link specified in the warning message clearly explains it.

When adding machines you can use machine FQDN. Visual Studio Team Services need no direct access to these test client machines. Having one release agent setup in the domain, which have access to the test client machines, as shown in Diagram 01, would be sufficient. Release agent machine can access the test client machines with domain credentials since it is inside the on-premise domain.

4. Create a release definition to create test client, and set to use an agent in the on premise domain. Configure on-premise agent for Visual Studio Team Services explained here.

5. Add Visual Studio Test Agent Deployment task to release definition. 

Specify the machine group name in Machines, created in step 3. Agent configuration user can be different from the user name provided for accessing the machines in machine group defined (same user used here). Agent configuration user is used to configure test agent, and test agent will run as this user. Check Interactive Process if Coded UI test needs to be run on the test clients.

This will auto deploy test agents to on-premise test clients in the machine group, from VS Team Services release.

Next post will explain how to run a coded UI test with the deployed agent.

Make Agent Queues - Usable to Team Project Users–TFS 2015 Build/Release Agent Pool Usage Access in Build/Release Definitions

Agent queue in a TFS 2015 build definition will not be listed for team project administrator, even though he can save a build definition.

To make agent pool available for the user creating a release or build definition in TFS 2015, you have to add to Agent queue users in the team project collection agent queues, “Agent Queue Users” role.

This will make the queue available to the build/release definitions. But this will enable access to all agent queues.

This happens because when you add to, it gets added to All queues level when you add as shown above.

To restrict to a given agent remove from “Agent Queue Users” role, of the collection and add only to the relevant agent queue.

This will make the user added only to the relevant queue, Agent Queue Users. Not to the collection role.

Only the permission granted agent queue is available for the user.

Environment Variables & TFS 2015 Build/Release Agents

If you install additional software in build server, or in a target server setup as a release server with build/release agent TFS 2015, you might run into the same issues specified below, which can be solved very easily.

The requirement

Need to run python and sphinx to generate documentation in the build server with the project.

The Issue

2016-06-28T08:56:24.4848312Z ##[error]The system cannot find the file specified.
2016-06-28T08:56:24.5316991Z The 'sphinx-build' command was not found. Make sure you have Sphinx
2016-06-28T08:56:24.5316991Z installed, then set the SPHINXBUILD environment variable to point
2016-06-28T08:56:24.5316991Z to the full path of the 'sphinx-build' executable. Alternatively you
2016-06-28T08:56:24.5316991Z may add the Sphinx directory to PATH.
2016-06-28T08:56:24.5316991Z If you don't have Sphinx installed, grab it from
2016-06-28T08:56:24.5316991Z http://sphinx-doc.org/

Checked environment variables for the build agent user and all were fine. All required python environment variables were there.

But when printed the path in the powershell script running the command to generate help, it seems environment variable path, were not having python paths.

The fix

It is just need a restart to build agents to reload environment variables.

This was an easy fix, but many times it can be missed, and would waste time investigating many other options.

Make Builds Run Faster

If your TFS builds are running too slow following might help, to reduce the time taken.

1. Set the build to enable parallel running and use multiple MSBuild processes say 4 processes. /m:4 /p:BuildInParallel=true

Setting /m /p:BuildInParallel=true  is better to let the number of MSBuild instances auto decide depending on build agent capability. More info http://www.hanselman.com/blog/FasterBuildsWithMSBuildUsingParallelBuildsAndMulticoreCPUs.aspx

With this set, the build server (agent) is still taking considerable time, and build only reduced by by maximum 5 minutes.  In the build agent noticed Antimalware protection is running, taking lot of CPU, while the build is going on.

2. In build agent windows defender or Endpoint Protection, exclude build agent installed folder and build working directories form scanning.

This gives a good performance boost to the builds.

Deploying to an Untrusted Domain–TFS 2015 Release Management Service

To setup an Agent (build/release agent), in an untrusted domain, follow the below steps.

1. Create local user in TFS App Tier machine say rmshadowagent

2. Navigate to TFS control panel using http://yourtfs:8080/tfs/_admin/_AgentPool and create a new agent pool. Then download the agent.zip file and keep it, to copy over to the target agent machine.

3. Select the agent pool and click on Roles. Add the local user (rmshadowagent)created in TFS App Tier to the “Agent Pool Administrators” role, by clicking Add and then “Add Windows user or group”.

4. Add the same user to “Agent Pool Services Accounts” role for the selected pool.

5. Copy over the agent.zip downloaded to the untrusted domain machine (agent machine). This machine should have access to you tfs url. This can be achieved via exposing your TFS via www. Then a machine with internet connection having access to TFS url can be an agent. Create folder for your agent. Example – C:\DeployAgent\Web, and extract agent.zip to it.

6. Create a working folder in agent machine for the agent.

7. In agent machine, if there is any credential remembered in the control panel for the TFS url, remove them. This is to make sure that you can provide credentials when configuring the agent.

8. In the agent machine create a user with the same user name say “rmshadowagent” and with the same password, you created in the TFS App Tier. (Add this user to Administrators group in agent machine to allow to perform any activity in the machine as a deployment agent).

9. Run a command prompt as administrator in the agent machine and change the directory to extracted agent folder. Then execute ConfigureAgent.cmd and provide the parameters to configure.

• Name for the agent
• TFS url – https://yourtfs/tfs
• Agent pool name
• Agent working directory path
• Install agent as service – y
• Instead of Network Service for the service user – provide .\rmshadowagent
• Provide the password for the rmshadowagent

10. Once click ok it gives below error. But the agent is configured and available to use in pool.

TF14045: The identity with type 'System.Security.Principal.WindowsIdentity' and
identifier 'S-1-5-21-1292816864-2021176197-253083057-1013' could not be found.

With this agent in untrusted domain build output gets downloaded, successfully confirming agent is running fine.

Configure Multiple Build/Release Agents in Single Build Machine–TFS 2015

To configure multiple TFS 2015 new build agents, in single build machine, follow the instructions below. It is recommended to limit the number of agents, in a build machine, to number of CPU cores it has.

Create folder structure for build agents, in the build machine something similar to below.

Download agent.zip from TFS Control panel, Agent pools tab.

Extract agent zip to each agent folder created for build agents.

Create agent working directories for each agent.

Run ConfigureAgent.cmd in each build agent directory and configure build agents.