Updates: With new updates to the Esri connector, you can now:
Esri Asset Federation and Data Connections are managed by site Administrators and Publishers using the Catalog Connector.
With the Esri ArcGIS server Connector, you can choose either to federate the layers to your Data & Insights catalog as external datasets or to create a data connection that downloads the layers from the ArcGIS server and creates both a map and dataset hosted on Data & Insights.
Federated layers will then become external datasets and will be searchable and discoverable within your Data & Insights catalog. Federating public Esri layers to your Data & Insights Catalog allows your users to discover and explore the rich data that you’re publishing through ArcGIS, alongside your other data assets.
Data connections download the data from your ArcGIS endpoint on a regular schedule and create public, published datasets hosted on Data & Insights. These datasets will also be searchable and discoverable within your Data & Insights catalog, but also provide your users the ability to download this geospatial data as well as API access via SODA.
- Your ArcGIS server must be a Map Server or a Feature Server. Other ArcGIS server types are not supported.
- The ArcGIS server needs to be publicly available
- The ArcGIS server needs to run on an https SSL cert (for more on this, see: http://resources.arcgis.com/en/help/install-guides/arcgis-server-web-adaptor-iis/10.2/index.html#//01nz00000027000000).
- The version of ArcGIS must be 10.0 or higher
- You'll need to have your ArcGIS Server Address to setup the connection. It will look something like this - https://<your domain>/arcgis/rest
- Feature layers must have REST query capabilities for the data connection to work properly
Setting Up a New Connection
To set up a connection you will navigate to the Esri ArcGIS and Data.JSON Connectors section from the Adminstration page on your site
On the Esri ArcGIS and Data.JSON Connectors page, select "Add New." This will bring up a screen where you can select Type = "Esri ArcGIS Server," the URL of the server, and a user-friendly display name. Please note that the Owner of assets set up in this connection will be the same as the user who sets the connection up.
Selecting Layers to Federate or Connect
When the connection is first established, none of your ArcGIS Map Server assets will be connected to your Data & Insights catalog by default. You’ll see this indicated in the “Connected assets” column of the connectors list as “0 assets connected”.
Administrators have the ability to curate the selection of connected assets and choose specific assets or groups of assets that you would like to make available to your users. Select “Edit” to curate the map layers that you want to federate to your Data & Insights catalog.
On the “Edit Connector” page, you’ll see three options for connection selection near the top. You’ll have the option to a) connect all assets in your ArcGIS Map Server as maps, b) federate all assets as external datasets, or c) to select a subset of assets and groups of assets to connect.
If you choose “Connect all assets as Maps” in the image below, then all the layers in your ArcGIS Map Server will be downloaded and a map and dataset will both be created for each one in your public Data & Insights catalog. If you choose this option then any new layers that are added to your ArcGIS server will automatically create new map visualizations and datasets and show up in your Data & Insights catalog upon the next sync.
If you choose “Connect all assets as External Datasets” in the image below, then all the layers in your ArcGIS Map Server will federate to your public Data & Insights catalog. If you choose this option then any new layers that are added to your ArcGIS server will automatically show up in your Data & Insights catalog upon the next sync.
If you choose “Connect a subset of assets,” then you’ll need to select the assets or groups of assets that you want to show in your public Data & Insights catalog as well as whether you would like to connect them as external datasets or as maps. With this method of connecting individual assets, anytime new layers are added to your ArcGIS server they will be discovered during the next sync but not added to the catalog automatically.
Once you’ve made your selection, press “save” to start the connection sync process for those assets.
Each time you edit the configuration and select more or different layers to connect, you’ll find the status of that sync process indicated as the “current status” on the "Manage Connectors" page. Once the sync status is marked “success” you’ll find those new layers in your Data & Insights catalog.
Set Update Frequency For Connected Layers
When connecting new layers, you can now set the sync time for each layer, with the option of both every day and every month.
After connecting a layer in the "edit connector" page, you will now be prompted with a drop-down next to the layer that contains two options: Syncs Daily and Syncs Monthly.
For assets that change often, select Syncs Daily. For assets that are more static, choose the monthly sync option.
Review and Sync Connected Assets
The Esri connector now also supports the ability to see the status of each layer as well as the option to immediately sync a layer. This view will give you a better understanding of which assets are connecting and which are failing.
You can review these options by selecting the link under the column Connected Assets.
The connected assets screen will give you detailed information regarding each connected layer showing the Connection Type, Status, Last attempted sync and last successful sync.
You can also choose to sync a single layer by choosing Sync Now in the Actions column.
For layers that have failed to connect, you will also have the option to retry the connection.
Finally, you also have the option to Sync all connections, by choosing Sync all now
Map Layers in the Catalog
When the Catalog Connector is established, map layers appear in your Data & Insights catalog as External Datasets or as Maps and Datasets depending on your selection.
Please note that ArcGIS servers are scheduled to sync data changes and update the map layers in your catalog as necessary, including selected metadata, once a day, but updates for very large servers may take longer than 24 hours. Additionally, dependent on available resources, updates to a connector may not happen every day. Each time it refreshes, changes that have taken place within your ArcGIS Map Server will show for each catalog listing.
Expected functionality when a map layer is federated to your Data & Insights catalog includes:
- If the layer is renamed in ArcGIS, the map and dataset names are changed in Data & Insights.
- If the layer is deleted in ArcGIS or moved to another ArcGIS server, the corresponding map and dataset are also deleted in your Data & Insights catalog.
- If the layer is moved or redeployed to another service or folder in the same ArcGIS server, the map and dataset appear as deleted in Data & Insights and a new map and dataset of the same name but with a different Unique ID (4x4) are added.
The overall sync will be successful, but failures on certain datasets will appear on the Activity Log page.
The Esri Connector will transfer the asset's title and make an attempt to bring the description if available - other fields will not be transferred. You can overwrite the title and description on the Data & Insights dataset that come from the Esri asset. However, if the title and description change on the Esri side, those changes will be migrated to the Data & Insights dataset on the next sync and overwrite the existing values. Metadata fields other than title and description on an Esri Connected Data & Insights dataset can be updated just like any other field and will not be changed by the Esri Connector.
With a daily frequency set, a layer is eligible to be synced after 24 hours from the last sync. The time of the sync will be dependent on when the layer is eligible and the current queue of all layers that are ready to be synced.
The Esri connection will remain active and any existing layers will continue to sync as expected.
If there is no data to update, the dataset will not update however since there was a sync to check for data, the sync is listed as successful.