DataDownload Tab

Introduction

The Download page is an optional component providing Extract Data and Email Task functionality for datasets. The data download interfaces with an ArcGIS Server Map Service and Geoprocessing service to provide the functionality to clip layers and download them. The data download customization does not come standard as part of the default Geoportal. The sample discussed in this topic should be used as a starting point to create a customized task that suits your organization's needs.

When downloading data, a user will draw the area they wish to download, select layers for download, select an output format for the data, and supply their E-Mail address. The Geoportal data download page will then use the input information as inputs to the data download geoprocessing service. When the request has been completed, an E-Mail message will be sent to the user with an attached ZIP file of their data.

The figure below shows the primary components of the Data Download functionality and the communication interfaces among them.

For Data Download to be functional in the Geoportal, the following must be present and configured:

• A map service to provide the view of the downloadable data
• A geoprocessing service to extract and process the selected data
• A mail server to send the clipped and zipped data to the requesting user via email
• The ArcGIS SOC account must have access to the data sources used in your map and geoprocessing services. Access must be Full Control (Read, Write, and Execute). If your data is in SDE, then the ArcSOC account must have these same privileges to the data. The ArcSOC account must also have Full Control privileges to the folder where the interim clipped output file is stored, and the arcgisserver folder.
• The ArcGIS Server machine must have access to the JavaScript API referenced in your gpt.xml file at the <interactiveMap jsapiUrl parameter. By default, this refers to the JavaScript API hosted on ArcGIS.com. If your geoportal will not have external internet access, you may have to deploy a local instance of the JavaScript API. Instructions for installing the JavaScript API locally are part of the ArcGIS JavaScript API product.
• Optional: Fully licensed Data Interoperability for ArcGIS Desktop and ArcGIS Server to provide certain additional output formats.

Configure DataDownload Tab

This section describes how to configure the Geoportal to enable data download from a custom map document using the ArcGIS 10 Extract Data and Email Task. This is a sample, and may require customization and testing by your organization to suit your organization's needs. Configuring Data Download for the Geoportal entails the steps below:

1. Prepare the geoprocessing task (Extract Data and Email Task)
2. Prepare the map service and geoprocessing service mxd file
3. Publish the map and geoprocessing services into ArcGIS Server
4. Configure the Geoportal to use these services in the Download tab

Prepare the geoprocessing task (Extract Data and Email Task)

The geoprocessing task used for the geoprocessing service is created by following instructions from the webhelp topic Extract Data and Email Task. After configuration according to this help topic, you should have your custom toolbox with the Extract Data and Email Task. A brief summary of the necessary steps to prepare the custom toolbox are below.

1. Open ArcMap with a blank document, to which you will add the layers that will be downloadable and your Extract Data and Email Task.
2. Enable the ArcToolbox window. Right click on the ArcToolbox icon and select Add Toolbox from the menu.
3. In the Add Toolbox dialog, click on either My Toolboxes or System Toolboxes, and select the new toolbox icon at the top right to create a new toolbox. Name this toolbox geoportal_data_download.
4. Navigate to the Server Toolstoolbox in the ArcGIS Toolbox window. Within the Server Tools toolbox, expand the Data Extraction toolbox.
5. Copy the following three items from Data Extractiontoolbox and paste them into your new toolbox:
   • Extract Data
   • Extract Data and Email Task
   • Send Email With Zip File Attachment
6. Now you will configure the email server settings in the Extract Data and Email Task. Right-click the Task from your geoportal_data_download toolbox and select Edit. The model window should appear.
7. Doubleclick the Send Email With Zip File Attachment box in the model. The input dialog box for the Send Email With Zip File Attachment tool should appear.
8. Enter values for From, Subject, Text, Max File Size, and SMTP Email Server. Click OK to persist the values you entered in the model.
9. Save and exit the model.
10. Save your map document whatever suits your organization. In this example, it will be called DataDownload.mxd.

Prepare the map service and geoprocessing service mxd file

From one map document will come both the map service and the geoprocessing service. This one map document - your DataDownload.mxd - must be prepared for the Extract Data and Email Task. This map document will be used to create a map and geoprocessing service within ArcGIS Server that will appear on the Download page of the Geoportal. The geoprocessing service will be used to clip and deliver the data to the end user.

Caution:

The Geoportal Download page can only display layers within one level of a Group layer in the Map Service. For example, below are three screenshots of the Table of Contents in ArcMap for three different mxd map documents. The one on the left would display all layers for download in the Geoportal. The one in the middle would also display all the layers, even though they are within a Group layer. The one on the right would not display the layers in the second Group layer, because they are a Group within a Group.

To prepare the DataDownload.mxd document do the following:

1. Add feature layers and/or raster layers that you want users to be able to clip and download.
2. In the ArcToolbox panel, expand the custom geoportal_data_download toolbox you created earlier.
3. Drag the Extract Data and Email Task model from that toolbox into your DataDownload map document's Table of Contents. Place the model at the top level of the Table of Contents, so it is listed as the first item and is above the other layers.
4. Right-click the Extract Data and Email Task model layer in the Table of Contents, and select Open.
5. Provide the inputs requested for your tool. Click OK to run.
6. Save the DataDownload.mxd.

Tip:

If you want to change the functions of the Extract Data and Email Task or other python scripts referenced in your document, you can make these changes and also change the model building without republishing mxd files. Republishing is needed only when you change input parameters to your model.

Publish the map and geoprocessing services into ArcGIS Server

You can use ArcGIS Server manager or ArcCatalog to publish the DataDownload.mxd file. Please read instructions from the webhelp topic Publishing geoprocessing services for the detailed process to publish into ArcGIS Server. At ArcGIS Server 10, ArcGIS Server will automatically generate a map service when you publish your geoprocessing service. Publish DataDownload.mxd as a geoprocessing service, and both the map and geoprocessing services will be available for integration into the geoportal.

Note:

After creating the two services, it is necessary to refresh the ArcGIS REST cache so they can be accessed by your geoportal. Refresh the ArcGIS REST cache by doing the following:

1. Open a web browser and type the following (substituting your ArcGIS Server machine): http://yourArcGISServer/arcgis/rest/admin
2. The ArcGIS REST API Admin page should appear. Log in to the page with your ArcGIS Server admin account.
3. After logging in, you will see the REST API Admin page. Select the Clear Cache Options link.
4. On the Clear Cache Options page, click the Clear Cache Now link.
5. Logout and close the browser.

Configure the Geoportal to use these services in the Download tab

The Extract Data and Email Task configuration is located in the <downloadData> element in your gpt.xml file. To configure this element, do the following:

1. Navigate to the \\geoportal\WEB-INF\classes\gpt\config folder and open the gpt.xml file in a text editor.
2. Scroll to the <downloadData> element.
3. Find the taskUrl attribute, and update it with the ArcGIS Server REST URL to your published DataDownload geoprocessing service. To get the REST URL to the ExtractDataTool, open a web browser and enter the following address (substituting your ArcGIS Server): http://yourArcGISServer/rest/services/. The DataDownload geoprocessing service should be listed in this list. Click on the Extract Data and Email Task link on that page. When the page redirects, copy the URL that is in the web browser and paste it in the taskUrl attribute, as shown below.

   taskUrl="http://yourArcGISServer/rest/services/DataDownload/GPServer/Extract%20Data%20and%20Email%20Task"
   

4. Find the mapServiceUrl attribute and update it with the ArcGIS Server REST URL to the published DataDownload map service. You can get the REST URL to the map service by going to the same ArcGIS REST services link as you did for the geoprocessing service (http://yourArcGISServer/rest/services/) and clicking on the link to your map service. In this example:

   mapServiceUrl="http://yourArcGISServer/rest/services/DataDownload/MapServer"
   

5. Find the mapServiceType attribute and update it with the value dynamic. This represents your data download map service's type, usually tiled or dynamic. Even if your map is a tiled map, enter dynamic as the value, as the display map will not display correctly if this value reads tiled.
6. Save the gpt.xml file.
7. Restart your geoportal web application.

There should now be a Download tab appearing on the interface, and your map services should be visible when the Download tab is selected.

Tip:

If you think the geoprocessing task is not working correctly, you can look at the message.xml log file in your %scratchworkspace%, e.g. C:\arcgisserver\arcgisjobs\extractdata_gpserver\{current job id}\scratch\.

Additional Output Formats

You will likely want to configure the Geoportal to offer different output file formats for the feature or raster layers in the Data Download interface. Feature output file formats are defined in the <features> tag in the gpt.xml file, and raster output file formats are defined in the <rasters> tag. Instructions for parameters within each <feature> and <raster> entry are described below.

Note:

For some output formats, the ESRI Data Interoperability must be installed and enabled in ArcGIS Desktop for the map service basemap and geoprocessing task basemap, and the Data Interoperability for ArcGIS Server must also be installed on the machine hosting ArcGIS Server. You may also need to adjust the Extract Data and Email ArcGIS Desktop task to handle the specified format. Because this customization is not included in the Geoportal general documentation, the Geoportal User Forum is a good place to find out how other users may have added additional data types for download.

1. The Data Interoperability uses Feature Manipulation Engine (FME) technology from SafeSoftware. To get the information that you will need to update the list of output formats, you will need to reference the FME website. The example below shows adding the ESRI ArcInfo format to the list of <features> feature output formats. Use a similar process to add a format to the <rasters> raster output formats.
   • Obtain the format identifier and file information by accessing the FEM website and clicking the link next to the format in which you are interested. This will redirect to a .pdf file giving additional details.
   • Find Format Identifier and Typical File Extensions. For ESRI ArcInfo Export, format identifier is "E00" and typical file extension is ".e00".
2. In the gpt.xml file, insert an additional <feature> element with the following attributes into the <features> element to include the supported output format.
   • key: {Format Identifier} - ({typical file extension}). In the ArcInfo example, it should be E00 - (.e00)
   • alias: optional - repeat the file extension.
   • resKey: assigns the key for referencing the display name as defined in gpt.properties for this format. Should be unique. In this example, resKey is catalog.download.feature.e00.
3. The new E00 entry has been added to the default list in the example below:

   <features>
   <feature key="File Geodatabase - GDB - .gdb" alias="gdb" resKey="catalog.download.feature.gdb"/>
   <feature key="Shapefile - SHP - .shp" alias="shp" resKey="catalog.download.feature.shp"/>
   <feature key="Autodesk AutoCAD - ACAD - .dxf" alias="dxf" resKey="catalog.download.feature.dxf"/>
   <feature key="Autodesk AutoCAD - ACAD - .dwg" alias="dwg"  resKey="catalog.download.feature.dwg"/>
   <feature key="Bentley Microstation Design (V8) - DGN_V8 - .dgn" alias="dgn"  resKey="catalog.download.feature.dgn"/>
   <feature key="E00 - (.e00) " alias = "e00" resKey="catalog.download.feature.e00"/>
   </features>
   

4. Open the gpt.properties file in your \\geoportal\WEB-INF\classes\gpt\resources folder
5. Add the display names for the added format in the # download data section, using your resKey described above as a guide. For example, we could add the key below to match the resKeyin the gpt.xml file:

   catalog.download.feature.e00= E00 - (.e00)

6. Save the gpt.properties file and the gpt.xml file. Restart the geoportal web application for you changes to take effect.