This section covers the registry keys and progress logic in the ACP.
The following keys are written to the registry under HKLM\Software\2pint Software\BITSTS\<DeploymentID>\<ContentID> registry path.
Registry value | Description |
---|---|
BytesFromPeers
Bytes from cache/peers for current job
BytesFromSource
Bytes from the server for the current job
BytesTotal
Total bytes of the download
Context
Error context
Description
Error description
EndTime
Date time when the job ended
Error
Last error during this download
ExitCode
The return code of the ACP, non zero means download failed
FilesTotal
Number of files in download
FilesTransferred
How many files have been transferred
JobId
The GUID of the BITS job
SequenceBytesFromPeersAfter
How many bytes the sequence had pulled from peers at the end of this step
SequenceBytesFromPeersBefore
How many bytes the sequence had pulled from peers at the beginning of this step
SequenceBytesFromSourceAfter
How many bytes the sequence had pulled from source at the end of this step
SequenceBytesFromSourceBefore
How many bytes the sequence had pulled from source at the beginning end of this step
SmallFilesCount
Number of small files in content that cannot be BranchCached
SmallFilesSize
Total size of the small files put together, this is the total size that cannot be peered and has to be downloaded from server
Source
The URL that is the source of this download
StartTime
The date time when this download started
Step
The step number of which this download is part of, note that multiple downloads can be done for the same step
StepName
The step name for this download, note that multiple downloads can be done for the same step
This page covers how to guarantee a certain download speed and at the same time protecting the source servers.
The OSD Toolkit is designed to allow Peer-to-peer (P2P) at background speeds as well as high speeds using the Turbo features. However, in some cases the speed is not enough or cannot be used as intended. A download from a unused local server will typically run at around 1000Mb/s, which is very fast, therefore a speed that we have gotten used to. If this server is shared across 10 concurrent downloads though, the theoretical speed is divided by 10, which is 100Mb/s. This speed is typically lower than the P2P speed that can be achieved, so when speed is this low, it makes sense to use P2P and not hammer the server.
The OSD Toolkit has advanced logic which is designed for allowing;
Use the server directly for locations where there are servers, still filling up BranchCache or other P2P technologies, but using them in a local mode instead, which improves speed.
Detect when speeds from local servers are low, then dynamically enabling peer-to-peer.
This allows for the following scenarios to be improved;
A single machine is built at a main office/build center with good local servers, but the machine is to be shipped to a location where no servers are available, so we want to fill up the BranchCache cache. In this case the machine uses local BranchCache mode, and just fills up it's own disk, but still downloading at close to wire speed.
In a situation where 500 machines are to be built at the same time, in this case the local P2P speeds will greatly outnumber any server capabilities, so here we dynamically enable peer-to-peer.
It allows the best of both worlds, fast server speeds when available and P2P when required as the server is busy.
Well connected is a term borrowed from the StifleR product set which means a "well connected site" either has good enough bandwidth to servers that are typically locally to the site itself. But even a site with 200Mb/s can be considered "well connected".
In our world, this is done by StifleR, in your world, we don't know, but you can still benefit from the functionality by setting the right task sequence values if you find out when you have local servers or not.
The local mode means the client does not query the network, instead it builds it's own cache over time, just like a read only file cache. Once the machine is switched to any other mode, this cached data can be server to other peers.
The following task sequence variables control this behavior.
Ifs the WellConnected2PS task sequece variable is set to "true" the binaries will treat this location as well connected by default and set BranchCache to local mode for any download. If the speed from the server goes below 100Mb/s after 5s of transfer time, the machine switches to distributed mode.
If the package has more than 100 files, local mode is automatically enabled as 100 files will never peer to peer well.
If this variable is set, the machine does not switch mode, and whatever mode was configured is kept.
Microsoft allows third parties to create an Alternate Content Provider (ACP) to allow downloads to be handed over to other technologies when the default Microsoft method may not work. The textbook definition from Microsoft:
"An Alternate Content Provider exposes a set of API calls and modifies the function of Data Transfer Services, such that an additional provider other than BITS can be used to retrieve the packages."
There are two types of alternative Content Provider (ACP) types:
System – Which requires an install and replaces BITS as the download engine
Task Sequence – Requires no installation on the system, and is only called during a Task Sequence, from a Task Sequence, using a Task Sequence variable
The OSD Toolkit uses a Task Sequence only ACP, for all other downloads, BITS is used in conjunction with BranchCache.
BranchCache is used for Task Sequences in the full OS when they are set to "Download all content locally before starting task sequence" as this means that they are scheduled via BITS. The BITS downloader supports BranchCache on all operating systems. An issue arises when a Deployment is set to "Download when required".
In this case only Enterprise clients can utilize BranchCache as these downloads use HTTP for content transfer and only these systems have native HTTP support for BranchCache. Although Enterprise can work without the ACP, we recommend the use of it for better speed and reliability.
Note: The necessary TS Deployment configuration is modified via a PowerShell script as this is not configurable directly in Configuration Manager. See the next section for details.
On Windows Pro machines custom configuration is required in order to make a "Download content locally when needed by running task sequence" distribution BranchCache enabled. This is configured via BITSACP.exe. This can be seamlessly integrated into any Configuration Manager 2012 task sequence deployment via the use of the Alternative Content Provider (ACP) support.
Note: If you want to use the same downloader on Enterprise in order to utilize the BITS bandwidth mechanism there is nothing stopping you. StifleR customers should always use the BITSACP to allow better monitoring. The BranchCache speed is also greatly improved by using the BITSACP with the turbo function.
As already mentioned, Windows Pro only supports BranchCache through BITS, or any other third party program communicating with BranchCache directly. This means that the "Download all content locally before starting task sequence" settings on Task Sequences Deployments will force the use of BranchCache as the content is downloaded by BITS before the task sequence is executed.
The snag with that setup is that ALL packages referenced by the task sequence are downloaded, even packages that have conditions on them, because the download engine does not evaluate these conditions. To get around this, you can set the Task Sequence deployment instead to: "Download content locally when needed by running task sequence", which makes the task sequence start and then only download packages when the conditions have been evaluated. The problem with this solution is that the Task Sequence downloader uses it’s own download engine rather than BITS.
This means two things:
Downloads are NOT throttled and likely to fail over limited bandwidth WAN pipes.
Downloads are NOT BranchCache enabled on Windows Pro machines as there is no native HTTP support for BranchCache.
The 2Pint Software BITS BC TS Download component BITSACP.exe addresses these issues by:
Acting as an Alternative Content Provider through the Extension provided by Microsoft to allow such third party download providers
Redirecting all Downloads to BITS with BranchCache support & bandwidth management
NOTE: When running a task sequence with an Alternative Content Provider, all downloads are first tried with the ACP. If this returns an error or if the data is not valid the client will automatically fall back to the regular built-in downloader.
Figure 16 A Windows Pro Machine is downloading packages in a task sequence using BITS & BranchCache. Although scheduled to run at 1Mb/s (WAN) it is caching local content at 61Mb/s from already cached content from its peers.
In an Operating System Deployment scenario, distribution point locations that are stored in task sequence variables are currently inaccessible to an Alternate Content Provider. This means that applications in System Center 2012 Configuration Manager that are written to use the Alternate Content Provider APIs are affected.
Microsoft recognized this short coming in functionality and issued an update. Information on this update and Microsoft Support for ACPs is available in the following knowledge base article:
http://support.microsoft.com/kb/2744420
The above article describes the update to support Alternate Content Providers in Task Sequences in System Center 2012 Configuration Manager.
To support an Alternate Content Provider in a Task Sequence, this update introduces the SMSTSDownloadProgram task sequence environment variable.
The SMSTSDownloadProgram task sequence environment variable is specified as follows:
A predefined environment variable that contains the path of the downloader program (.exe file) will be set in the SMSTSDownloadProgram task sequence environment variable.
During the download process, the task sequence download library will check the environment variable to verify that there is a defined downloader program. If there is, then this program will be used to perform the download.
The BITSACP.EXE is available in the boot.wim file generated by WinPEGen.exe so there is no need to copy it into the boot image. The "BCEnabler.exe Move" command also moves this to \Windows\System32 for the deployed Operating System.
The following command line parameters are passed to the external downloader program:
Package ID
Absolute path of the download folder
Notes
The Configuration Manager client expects at least one location. The location will be passed to the Alternate Content Provider as it can use the Configuration Manager Distribution Point in certain situations.
The Alternate Content Provider can return a fallback code or an error code. When a nonzero code is returned, the Configuration Manager client will fall back to Content Transfer Manager and Data Transfer Service.
An Alternate Content Provider will work only inside a Task Sequence. It cannot be called outside of a task sequence.
After all the explanation and environmental tweaks the actual configuration required to kick off the ACP is done in minutes.
First, find the appropriate place in the Task Sequence to add the step that enables the Task Sequence ACP. To ensure that all content is downloaded using the new downloader this would normally be one of the first actions, before any Packages are referenced. When you have determined the best place add a "Set Task Sequence Variable" action from the Add menu. Then give it a name of your liking and set the Task Sequence Variable name to: SMSTSDownloadProgram and set the Value to BITSACP.exe.
If you want to use the ACP for task sequences set to "Download on demand" for already running operating systems you must ensure that the executable is available on the machines on which it is to be run. This can be achieved by either putting the executable in a separate package/program which is then distributed to all machines before reimaging or by copying it to the machine as a part of the task sequence. In order to be safe you can do both - distribute the executable to all systems and also add a step in the task sequence to copy it to Windows if it’s missing.
When the Task Sequence executes, the downloader executable will be called and will start the download. The downloader updates the Task Sequence progress bar accordingly. The following picture shows it in action:
Figure 17 The downloader in action. This shows both the number of files and number of Bytes left to download.
The Downloader creates BITS jobs for each package that is requested by the task sequence. This can be verified by running BITSADMIN.EXE or PowerShell cmdlets to list the download jobs.
Figure 18 BITSADMIN listing active downloads as the Task Sequence is running.
Figure 19 The event logged by BITS after a download. Note the BranchCache data information.
Not logging as such but when BITSACP and BCEnabler run they do write some very useful information to the registry. This happens in WinPE as well as in the full operating system.
The information is written to the HKLM\Software\2Pint Software key.
Under the BCEnabler sub key the following values are written:
BCPort - Port for BranchCache to operate on. Default will show 80 (decimal).
PreferedContentInformationVersion - Version of BranchCache. Unless specified this will show (2) for a Win8 or later WinPE or (1) for a Win7 (3.1) version of WinPE.
CacheRootFolder - the directory under which the cache data is stored.
Figure 20 The port and path to the files being used by BranchCache. These are updated at Boot and at Enable and Move steps.
The ACP writes package information to a separate key for each located under the BITSTS key.
Figure 21 Generic information written under the main key. Each package then has its own key underneath the BITSTS key in HKLM\Software\2Pint Software\BITSTS
For each package the following information from the BITS job is written to a key underneath the BITS registry key. The name of the value is in the format of the ContentID such as R1101234.
The data stored is:
Number of bytes from the DP
Number of bytes downloaded from peers (also de-duplicated data)
Total bytes of the content
Total bytes downloaded
Number of files in the content
Number for files transferred in the content
NOTE: This data may be used to create custom or other action in the task sequence.
Figure 22 Data and statistics for a package being downloaded. Note that the BranchCache statistics are not available until the download has completed.
Troubleshooting
The status of a package can be monitored directly from the registry as it downloads.
If no data is coming from BranchCache check clients and firewall settings.
From the command prompt enter
>netsh.exe BranchCache show status all
while the download is happening. If data is being loaded into the cache but not from BranchCache Peers it is likely to be a firewall/port issue. (Assuming that the other clients have the required data cached)
To view the attempted network connection type "netstat –n", it should list the attempted BranchCache connections.
Customizing the Boot Image with Non-Default (EN-US) language options is supported but requires access to source media with the same language.
For more information on how to create a localized image please review:
https://msdn.microsoft.com/en-us/library/windows/hardware/dn938382(v=vs.85).aspx
Example of command lines using DISM.EXE to create a Swedish WinPE version with the language ID of: sv-se
copype amd64 c:\temp\win10amd64
dism.exe /mount-wim /wimfile:C:\temp\win10amd64\media\sources\boot.wim /index:1 /mountdir:C:\temp\win10amd64\mount
Dism /Add-Package /image:C:\temp\win10amd64\mount /PackagePath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\sv-se\lp.cab"
Dism /image:C:\temp\win10amd64\mount /set-allIntl:sv-se
REM Dism /image:C:\temp\win10amd64\mount /Set-UILang:sv-se
REM Dism /image:C:\temp\win10amd64\mount /Set-SysLocale:sv-se
REM Dism /image:C:\temp\win10amd64\mount /Set-UserLocale:sv-se
REM Dism /image:C:\temp\win10amd64\mount /Set-InputLocale:041d:0000041d
REM Dism /image:C:\temp\win10amd64\mount /Set-TimeZone:"W. Europe Standard Time "
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\winpe-wmi.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\winpe-scripting.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\winpe-wds-tools.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\WinPE-SecureStartup.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\sv-se\WinPE-SecureStartup_sv-se.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\sv-se\WinPE-WMI_sv-se.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\sv-se\WinPE-Scripting_sv-se.cab"
dism.exe /image:C:\temp\win10amd64\mount /add-package /packagepath:"C:\Program Files (x86)\Windows Kits\10\Assessment and Deployment Kit\Windows Preinstallation Environment\amd64\WinPE_OCs\sv-se\WinPE-WDS-Tools_sv-se.cab"
dism.exe /unmount-wim /mountdir:C:\temp\win10amd64\mount /commit
Running the generator is the same as a regular English version, but it will show the non-default language. If the language is not matching the WinPE version it will error.
This section describes how and what the OSD Toolkit turbo feature does
The OSD Turbo feature is a way to speed up deployments (typically OSD) related content transfers. Normally these are done via BITS, which can run at a slower rate than what is optimal. It's a numbers game right, sometimes people just want to go faster. A detailed explanation on this is available here: https://2pintsoftware.com/the-updated-osd-toolkit-turbo-feature/
The TL;DR version is that we call the native BranchCache API's with bigger buffers to get better speed.
The CPU of both pulling and service source machines matter. The faster the CPU's, the faster the download is. But since the data has to be decrypted, it will peg your CPU of the machine downloading more than the machine serving the content.
Use maximum 5 files in your package source (The Turbo only transfers 5 files). But the Turbo still runs for the largest 5 files for any package.
Keep the size over 50MB per file, Turbo will not run for files smaller than 50MB.
Don't include small .txt files in the source, this will lead to the Turbo downloading the big files and then using BITS to transfer the small files after, which is not so fast.
Simple, set a variable named “2PintTurboLicense” to the license key that you got from us. The if you want to disable it, set the license key variable to a blank value (which removes the value).
The Turbo feature writes a log file and also updates the registry keys with information about duration etc. The following registry keys and values are written when the Turbo components runs, they can be used for reference to troubleshoot BranchCache issues.
All TS keys are created under the HKLM\Software\2pint Software\BITSTS\<DeploymentID>\<ContentID> registry path. Each folder is has the following key registry values and return codes. Where <FileId> is mentioned, this is the local file name for the file inside the content.
Registry value | Description |
---|---|
Value: TurboReturnCode_<filename>
The following known return codes are available:
Any other errors than above typically indicates an issue with the error code as an indidation of what went wrong.
Return code | Description |
---|---|
TurboDuration
Total number of seconds the Turbo took to complete for all files (3.0.5.0 and prior this is for the last file)
BytesTotal_<FileId>
How many bytes transferred via turbo for this file
TurboSpeed
The speed of the last turbo file
TurboProgress
Progress in bytes of the last file
TurboReturnCode_<FileId>
The return code per file id
BytesTotalTurbo
Total bytes transferred successfully via Turbo
0
No errors, all content downloaded via Peer-to-peer
1
General error, most likely certificate based error
4052
The requested data cannot be found in local or peer caches. Typically the content is not on any peers, or only partially there.
4053
The server did not calculate or have the hash available at this point
4062
The requested version is not supported. Most likely you are running on Windows 7.
4064
Not licensed, there is no license for either BranchCache or the 2Pint Software OSD Toolkit Turbo feature.