MSIT internal Opalis adoption - Using Opalis to Move Files Across Untrusted Domains

  • Article

This post was written by Marian Stanescu

I work in a small team at Microsoft that manages and monitors about 5,000 servers used to run a few of the larger Microsoft online sites. To process our data, we constantly need to move files across untrusted domains.  We used to do this with a 3rd party tool but it required its own infrastructure and management. Working with Opalis, we found a way to move those files using Opalis workflows. As this improvement has saved us a lot of time, I thought I would share the architecture and steps with this community. 

We have a simple Opalis infrastructure where action servers in various domains are connected to the same database – through SQL authentication where needed.  Here’s the steps we took to automate this process: 

1. Create the file in one of the domains and save it as a txt file (same method could be used to transfer binary file however encoding needs to be used).

2. Start a workflow in the source domain, read the file from the beginning to the end (fig. 1) and flatten the output with option “separate with line breaks” (fig. 2). Put the data as a blob on the Opalis databus.

Fig. 1

clip_image002

Fig. 2

clip_image004

3. Within the initial workflow call into another workflow that is scheduled to run on the destination domain (that is the workflow is specifically scheduled to run on an action server that exists in the destination domain) and pass the file data blob as an input to this second workflow (fig. 3). To be able to rebuild the file we also pass in the original file name.

Fig. 3

clip_image006

4. Save the blob into a SQL DB along with the file name. We use a column of type NTEXT in SQL to save the blob.

5. In the destination domain, we read the file name and the blob from SQL and restore the file to the original content on the disk (fig.4).

Fig. 4

clip_image008

As a note - the “string” parameter type (for Custom Start Object that we use in the policy receiving the blob) is stored by Opalis in an NTEXT SQL type column that can hold up to 2 GB data. 

We have used this approach to move .txt  files up to 1 MB. Using this solution, we have been able to:

1. Gain runtime access to business data across untrusted domains within one Opalis workflow

2. Move / Copy files across untrusted domains with Opalis

3. Leverage out-of- box functionality versus going to third-party tools

If you’re following me this far, chances are you run into this scenario as well. It’s pretty common, and also pretty simple to fix.  Below’s a Visio that sums it all up. 

opals diagram

  • Adam Hall-blog    Adam Hall

  •    Snr. Technical Product Manager

  •    image

 

Send me an e-mailFollow me on TwitterConnect with me on LinkedIn