Esri Asset Federation and Data Connections

Updates: With new updates to the Esri connector, you can now:

• Set Update Frequency For Connected Layers
• Review and Sync Connected Assets

Esri Asset Federation and Data Connections are managed by site Administrators and Publishers using the Catalog Connector.

With the Esri ArcGIS server Connector, you can choose either to federate the layers to your Data & Insights catalog as external datasets or to create a data connection that downloads the layers from the ArcGIS server and creates both a map and dataset hosted on Data & Insights.

Federated layers will then become external datasets and will be searchable and discoverable within your Data & Insights catalog. Federating public Esri layers to your Data & Insights Catalog allows your users to discover and explore the rich data that you’re publishing through ArcGIS, alongside your other data assets.

Data connections download the data from your ArcGIS endpoint on a regular schedule and create public, published datasets hosted on Data & Insights. These datasets will also be searchable and discoverable within your Data & Insights catalog, but also provide your users the ability to download this geospatial data as well as API access via SODA.

Technical Requirements

• Your ArcGIS server must be a Map Server or a Feature Server. Other ArcGIS server types are not supported.
• The ArcGIS server needs to be publicly available
• The ArcGIS server needs to run on an https SSL cert (for more on this, see: http://resources.arcgis.com/en/help/install-guides/arcgis-server-web-adaptor-iis/10.2/index.html#//01nz00000027000000).
• The version of ArcGIS must be 10.0 or higher
• You'll need to have your ArcGIS Server Address to setup the connection. It will look something like this - https://<your domain>/arcgis/rest
• Feature layers must have REST query capabilities for the data connection to work properly

Setting Up a New Connection

To set up a connection you will navigate to the Esri ArcGIS and Data.JSON Connectors section from the Adminstration page on your site

On the Esri ArcGIS and Data.JSON Connectors page, select "Add New." This will bring up a screen where you can select Type = "Esri ArcGIS Server," the URL of the server, and a user-friendly display name. Please note that the Owner of assets set up in this connection will be the same as the user who sets the connection up.

Selecting Layers to Federate or Connect

When the connection is first established, none of your ArcGIS Map Server assets will be connected to your Data & Insights catalog by default. You’ll see this indicated in the “Connected assets” column of the connectors list as “0 assets connected”.

Administrators have the ability to curate the selection of connected assets and choose specific assets or groups of assets that you would like to make available to your users. Select “Edit” to curate the map layers that you want to federate to your Data & Insights catalog.

On the “Edit Connector” page, you’ll see three options for connection selection near the top. You’ll have the option to a) connect all assets in your ArcGIS Map Server as maps, b) federate all assets as external datasets, or c) to select a subset of assets and groups of assets to connect.

If you choose “Connect all assets as Maps” in the image below, then all the layers in your ArcGIS Map Server will be downloaded and a map and dataset will both be created for each one in your public Data & Insights catalog. If you choose this option then any new layers that are added to your ArcGIS server will automatically create new map visualizations and datasets and show up in your Data & Insights catalog upon the next sync.

If you choose “Connect all assets as External Datasets” in the image below, then all the layers in your ArcGIS Map Server will federate to your public Data & Insights catalog. If you choose this option then any new layers that are added to your ArcGIS server will automatically show up in your Data & Insights catalog upon the next sync.

If you choose “Connect a subset of assets,” then you’ll need to select the assets or groups of assets that you want to show in your public Data & Insights catalog as well as whether you would like to connect them as external datasets or as maps. With this method of connecting individual assets, anytime new layers are added to your ArcGIS server they will be discovered during the next sync but not added to the catalog automatically.

Once you’ve made your selection, press “save” to start the connection sync process for those assets.

Each time you edit the configuration and select more or different layers to connect, you’ll find the status of that sync process indicated as the “current status” on the "Manage Connectors" page. Once the sync status is marked “success” you’ll find those new layers in your Data & Insights catalog.

Set Update Frequency and Visibility For Connected Layers

When connecting new layers as a data connection, you can now set the sync time for each layer, with the option of both every day and every month.

After connecting a layer in the "edit connector" page, you will now be prompted with two drop-downs next to the layer. These options allow you to set how often they sync and the publication level of the asset. 

Update Frequency

The available options for update frequency are Syncs Daily and Syncs Monthly.

For assets that change often, select Syncs Daily. For assets that are more static, choose the monthly sync option.

Visibility

On Open Data sites these options are Public and Private.

On Enterprise Data sites the options are Public, Private, and Internal

 

Review and Sync Connected Assets

The Esri connector now also supports the ability to see the status of each layer as well as the option to immediately sync a layer. This view will give you a better understanding of which assets are connecting and which are failing.

You can review these options by selecting the link under the column Connected Assets.

The connected assets screen will give you detailed information regarding each connected layer showing the Connection Type, Status, Last attempted sync and last successful sync.

You can also choose to sync a single layer by choosing Sync Now in the Actions column.

For layers that have failed to connect, you will also have the option to retry the connection.

Finally, you also have the option to Sync all connections, by choosing Sync all now

Map Layers in the Catalog

When the Catalog Connector is established, map layers appear in your Data & Insights catalog as External Datasets or as Maps and Datasets depending on your selection.

Please note that ArcGIS servers are scheduled to sync data changes and update the map layers in your catalog as necessary, including selected metadata, once a day, but updates for very large servers may take longer than 24 hours. Additionally, dependent on available resources, updates to a connector may not happen every day. Each time it refreshes, changes that have taken place within your ArcGIS Map Server will show for each catalog listing.

External Link assets can also be configured with different permission levels

On Open Data sites these options are Public and Private.

On Enterprise Data sites the options are Public, Private, and Internal

Expected functionality when a map layer is federated to your Data & Insights catalog includes:

• If the layer is renamed in ArcGIS, the map and dataset names are changed in Data & Insights.
• If the layer is deleted in ArcGIS or moved to another ArcGIS server, the corresponding map and dataset are also deleted in your Data & Insights catalog.
• If the layer is moved or redeployed to another service or folder in the same ArcGIS server, the map and dataset appear as deleted in Data & Insights and a new map and dataset of the same name but with a different Unique ID (4x4) are added.

FAQs