Here at Simply Migrate we are always looking for ways to improve our clients migration experience and clarify the technologies in what can sometimes be a complex, acronym flooded industry. Fast Transfer Stream, on the face of it, is one of those closed black box technologies that gets talked about during Exchange migrations, but there is very little practical information about getting started with FTS. This blog aims to give the reader a practical technical understanding of FTS enabling them to discuss, inspect, parse or write streams.
Most people’s first exposure to FTS is when evaluating how to achieve full fidelity imports and exports from Exchange. This quickly leads to the Exchange Web Services (EWS) functions UploadItems and ExportItems. The UploadItems Request defines a Data element of type base64Binary, with a description of “The Data element contains the data of a single exported item or an item to upload into a mailbox.”, this implicitly tells you that the data element can be retrieved from ExportItems or could be an item from a different source.
But what is the data structure and what are the practical uses and limitations? How can the data be parsed? Is it possible to create my own data string?
If you haven’t guessed the answer is FTS; both the UploadItems and ExportItems work with a Base64 encoded binary Fast Transfer Stream. Microsoft kindly do provide the technical details of the inner working of the stream in [MS-OXCFXICS]: Bulk Data Transfer Protocol but there is a lot of implied knowledge and skill to do anything functional with the information.
The next few steps will give you some hands-on experience with working with FTS, provide a base to work with the finer details of the specification and highlight tools and techniques that will make your task a little bit easier.
Step 1 – Let’s get a FTS text string of a message
The first thing needed is the Base64 text of a message that we want to work with, as noted above we can quite easily retrieve this using the ExportItems EWS call. One way to do this is to use EWS Editor which gives you the ability to make EWS calls without having to write any code, it also comes with loads of handy examples so you don’t need to worry about the request/response formats.
Open EWS Editor and connect to a mailbox (Exchange Service), once connected you can attach to the MsgFolderRoot and browse to the folder where the item you want to work with resides. Once you have the folder, click on it and find your message, double-clicking your message will then display a list of properties, the important one for ExportItems is the Id property. Find the Id property and double-click, record the value of the UniqueId as displayed in the “Value (Default)” field of the “Property Definition” (note the ChangeKey is not required).
Once you have your message Id you can fire up the EWS Post tool (Tools -> EWS POST) within EWS Editor. You’ll need to enter your credentials and press “Load Example” and select “ExportItems.xml”. Delete all but one of ItemIds in the “Request Text” and then modify the remaining ItemIds to match the previously recorded message Id.
Finally press Run and if everything is correct you should be rewarded with a Data element within the “Response Text” containing the Base64 string of the FTS for your selected message.
Step 2 – Decode the Base64 string and save the binary content
Now that we have the Base64 string we need to decode it to get the binary content of the FTS. Use an online Base64 decoder such as motobit to convert the Base64 string into a binary file.
Ensure that you use the iso-8859-1 character set for decoding, using a different character set will cause the resulting binary to be unreadable in the next step.
After pressing the “Convert the source data” button you will have a file containing the binary of the FTS stream.
Step 3 – Inspect the FTS binary to view structure and properties of your message
Open your binary file with 010 Editor and then open and run the FTS template, in the “Template Results” you should see the structure and data of your selected message.
You have just decoded your first Fast Transfer Stream message property, so as you can see with some prodding and poking getting to the heart of an FTS stream of data can be easier than what reading a Microsoft protocol specification document might make one think. 🙂 However, what you do with that in your software now is up to you!