Skip to main content

Join additional machines to an ArcGIS Data Pipelines Server site

Multiple-machine ArcGIS Data Pipelines Server sites allow you to support more active data pipeline authors working in the editor, or more scheduled task runs simultaneously. If you have many active data pipeline authors or plan on scheduling many task runs, or if your single machine site's resources are inadequate for your users' needs, expanding to a multiple-machine site could be beneficial.

All machines in an ArcGIS Data Pipelines Server site must have the same version and license file. On the first machine you set up, you will need to create a ArcGIS Data Pipelines Server site. You can then join any additional ArcGIS Data Pipelines Server machines to the site.

Prepare to join an ArcGIS Data Pipelines Server

When you create an ArcGIS Data Pipelines Server site, you specify locations for the configuration store and server directories. In a multiple-machine site, you must share the configuration store and server directories so that the other machines can access them. If you did not specify a custom location when creating the site, by default these directories can be found at c:\arcgisdatapipelinesserver.

When you create an ArcGIS Data Pipelines Server site, you specify locations for the configuration store and server directories. In a multiple-machine site, you must share the configuration store and server directories so that the other machines can access them. If you did not specify a custom location when creating the site, by default these directories can be found at <ArcGIS Data Pipelines Server installation directory>/datapipelines/usr/.

Install and configure a multiple-machine ArcGIS Data Pipelines Server site

When setting up a multiple-machine ArcGIS Data Pipelines Server site, additional steps are required.

  1. Verify that your machine meets the system requirements for ArcGIS Data Pipelines Server.

    The installed version of ArcGIS Data Pipelines Server and its license file must be the same on each machine. Enter the same ArcGIS Data Pipelines Server account during each installation. It's recommended that you use a domain or Active Directory account, but if you choose a local account, it must exist on each machine with an identical name and password combination. Keep in mind that the account you specify does not have to be part of the Administrators group.

  2. Before you can join a machine to an ArcGIS Data Pipelines Server site, you must first have an existing site to join with. To create the initial site, see Create an ArcGIS Data Pipelines Server site.

  3. Grant read and write access on both the directories folder and the configuration store folder of the existing site to the ArcGIS Data Pipelines Server account. Make sure to set permissions in both the Sharing and Security tab of File Explorer.

    Note:

    ArcGIS Data Pipelines Server does not support shared network directories that are hidden or that use special characters (such as $).

    If using a UNC path for directories and config-store, you must escape any backslashes (\) when designating them. For example, if the path is \\mymachine\config-store, specify \\\\mymachine\\config-store.

    • If you are using a group managed service account (gMSA) for the ArcGIS Data Pipelines Server account, you must allow service accounts to have full control permissions on the Sharing and Security tabs. From the Permissions editor, click Add, and open the Object Types dialog box. Check the box for Service Accounts.

  4. Install ArcGIS Data Pipelines Server on the machine or machines that will join the existing site.

  5. Join each additional machine to the site using the configuration wizard by following the steps below. If you prefer to join the site from the command line, see the next step.

    1. From the first additional machine, open the ArcGIS Data Pipelines Server Configuration Wizard.

    2. Click Join Existing Site.

    3. Enter the site URL and credentials for your ArcGIS Data Pipelines Server site as created in step 2.

    4. Repeat for each additional machine you want to join to the site.

    When additional machines are joined to the site, they are considered equal in the site; there is no primary machine.

  6. Alternatively, join each additional machine to the site using the command line utility as follows:

    1. Ensure you're signed in to the machine with the ArcGIS Data Pipelines Server account.

    2. Open a command prompt and run the join-site.bat utility, which is located at <ArcGIS Data Pipelines Server install directory>\tools\joinsite. The following are the command line parameters for the utility:

    • -u or --username—The user name of the primary site administrator. The user name cannot contain these characters: \|/:*?<>.

    • -p or --password—The password for the primary site administrator. The user name cannot contain these characters: \|/:*?<>.

    • -f or --file—The configuration store connection file for the join-site utility.

    • -s or --siteUrl—The URL of the existing ArcGIS Data Pipelines Server site to join, in the format https://datapipelinesserver.example.com:14443/arcgis/admin.

    • -h or --help—Displays command line help and exits.

    The following is an example command for the utility:

    <ArcGIS Data Pipelines Server install directory>\tools\joinsite\join-site.bat -u PSAusername -p PSApassword -s https://datapipelinesserver.example.com:14443/arcgis/admin

    If the account credentials and site URL are entered correctly, the utility will join the machine to the ArcGIS Data Pipelines Server site. Repeat this step for each additional machine.

Tip:

When you join a machine to a site, logs at the Verbose level are written to <ArcGIS Data Pipelines Server installation directory>\usr\logs\<machine name>\server. If the join site fails, you can open the logs with a text editor to get more information about the problem.

Once the join site is complete, the logs are moved to <ArcGIS Data Pipelines Server installation drive>\arcgisdatapipelinesserver\logs and the log level is set to Warning.

  1. Verify that your machine meets the system requirements for ArcGIS Data Pipelines Server.

    Make sure you install using the same operating system user on both machines. The user account used to install ArcGIS Data Pipelines Server will be the account used to run the software. Group users and all others must have at least execute permissions to the ArcGIS Data Pipelines Server installation directory. Note that installing ArcGIS Data Pipelines Server with the root account is not supported.

  2. Before you can join a machine to an ArcGIS Data Pipelines Server site, you must first have an existing site to join with. To create the initial site, see Create an ArcGIS Data Pipelines Server site.

  3. Grant at least 700 permissions on both the directories folder and the configuration store folder of the existing site. These permissions should be granted to the ArcGIS Data Pipelines Server account.

    Note:

    ArcGIS Data Pipelines Server does not support shared network directories that are hidden or that use special characters (such as $).

  4. Install ArcGIS Data Pipelines Server on the machine or machines that will join the existing site.

  5. Join each additional machine to the site using the configuration wizard by following the steps below. If you prefer to join the site from the command line, see the next step.

    1. From the first additional machine, open the ArcGIS Data Pipelines Server Configuration Wizard.

    2. Click Join Existing Site.

    3. Enter the site URL and credentials for your ArcGIS Data Pipelines Server site as created in step 2.

    4. Repeat for each additional machine you want to join to the site.

    When additional machines are joined to the site, they are considered equal in the site; there is no primary machine.

  6. Alternatively, join each additional machine to the site using the command line utility as follows:

    1. Ensure you're signed in to the machine with the ArcGIS Data Pipelines Server account.

    2. Open a command prompt and run the join-site.sh utility, which is located at <ArcGIS Data Pipelines Server install directory>/datapipelines/tools/joinsite. The following are the command line parameters for the utility:

    • u or --username—The user name of the primary site administrator. The user name cannot contain these characters: \|/:*?<>

    • -p or --password—The password for the primary site administrator. The user name cannot contain these characters: \|/:*?<>

    • -f or --file—The configuration store connection file for the join-site utility.

    • -s or --siteUrl—The URL of the existing ArcGIS Data Pipelines Server site to join, in the format https://datapipelinesserver.example.com:14443/arcgis/admin.

    • -h or --help—Displays command line help and exits.

    The following is an example command for the utility:

    <ArcGIS Data Pipelines Server install directory>/datapipelines/tools/joinsite/join-site.sh -u PSAusername -p PSApassword -s https://datapipelinesserver.example.com:14443/arcgis/admin

    If the account credentials and site URL are entered correctly, the utility will join the machine to the ArcGIS Data Pipelines Server site. Repeat this step for each additional machine.

Tip:

When you join a machine to a site, logs at the Verbose level are written to <ArcGIS Data Pipelines Server installation directory>/datapipelines/usr/logs/<machine name>/server. If the join site fails, you can open the logs with a text editor to get more information about the problem.

Once the join site is complete, the logs are moved to <ArcGIS Data Pipelines Server installation drive>/datapipelines/usr/logs and the log level is set to Warning.

Once all the machines are joined to the site, the remaining setup process is the same as for single-machine sites. Proceed to install and configure ArcGIS Web Adaptor with your site.

Back to top