Poll an ArcGIS Server for Features

The Poll an ArcGIS Server for Features input connector can be used to poll an ArcGIS Enterprise or ArcGIS Online map or feature service layer for feature records to process them as event records.

Usage notes

Keep the following in mind when using the Poll an ArcGIS Server for Features input connector:

• Use this input connector to retrieve features from a map or feature service layer.
• This input connector pairs the Feature JSON inbound adapter with the Feature Service inbound transport. The Feature JSON inbound adapter specifically interprets feature JSON. For more information, see Geometry objects.
• ArcGIS Enterprise, ArcGIS Online, and ArcGIS Server(stand-alone) map and feature services are supported.
• Service layers can be referenced using the Browse to Layer option or by specifying a URL with the Service Layer URL option. Both methods require that a registered data store connection first be established.
• Like all input connectors, a GeoEvent Definition is required. A GeoEvent Definition identifies the schema of event records being received. For more information, see GeoEvent Definitions.
• Data values from event attribute fields not included in a GeoEvent Definition are ignored.
• This input connector periodically polls a specified map or feature layer at a specified interval. As with any web server or web service, excessive polling intervals—for example, every second—are not recommended.
• A simple WHERE clause is used to identify the feature record set returned with each poll of the map or feature service. The default is to include all records in the polled feature record set, for example, WHERE 1=1. The default WHERE clause can be modified to include specific attribute values and logic conditions.
• This input connector supports spatial filters that use a specified geometry and spatial relationship, for example, polygon intersects.
• This input connector supports the optional deletion of polled records from the map or feature layer being polled.
• This input connector supports a unique capability, get incremental updates, to poll for newly added or updated feature records using ObjectID or Timestamp values.
• There are four options to choose from to configure incremental polling. You can choose to use the feature record object identifier, a timestamp in a since last-received context, a timestamp in an interval between last query until now context, or a timestamp which overlaps with the system's current date and time.
• For more information on polling incremental updates, see the following resources on the Esri Community:
  • Poll an ArcGIS Server for Features: Methods used to identify incremental updates
  • Polling Feature Services for "Incremental Updates"
  • Question regarding "Incremental Update" workarounds, custom components
  • Using a partial GeoEvent Definition to update feature records

Parameters

The following are the parameters for the Poll an ArcGIS Server for Features input connector:

Parameter	Description
Name	A descriptive name for the input connector used for reference in GeoEvent Manager.
Registered server connection	An ArcGIS Server, ArcGIS Enterprise, or ArcGIS Online connection registered with ArcGIS GeoEvent Server as a data store. Registered server connections cache information about map/feature services, their layers, and layer properties.
Reference to Layer Type	Defines the options for referencing a layer. Browse to Layer—Reference a service layer by browsing the folders of a registered data store. Service Layer URL—Reference a service layer by specifying the URL of the shared service layer.
Folder (Conditional)	An ArcGIS Server services folder, ArcGIS Enterprise portal, or ArcGIS Online content item folder. The parameter is shown when the Reference to Layer Type parameter is set to Browse to Layer and is hidden when set to Service Layer URL.
Service Name (Conditional)	The name of the map/feature service from which you want to poll feature records. The parameter is shown when the Reference to Layer Type parameter is set to Browse to Layer and is hidden when set to Service Layer URL.
Layer (Conditional)	The map/feature service feature layer through which feature records are accessed. The parameter is shown when the Reference to Layer Type parameter is set to Browse to Layer and is hidden when set to Service Layer URL.
Service Layer URL (Conditional)	The URL to a service layer. Remarque : A data store connection must be established when referencing a service layer by URL. The service layer must either be publicly accessible or shared with the user whose credentials establish the data store connection. The parameter is shown when the Reference to Layer Type parameter is set to Service Layer URL and is hidden when set to Browse to Layer.
Create GeoEvent Definition	Specifies whether a new or existing GeoEvent Definition should be used for the inbound event data. A GeoEvent Definition is required for GeoEvent Server to understand the inbound event data attribute fields and data types. Yes—A new GeoEvent Definition will be created based on the schema of the first event record received. No—A new GeoEvent Definition will not be created. Select an existing GeoEvent Definition that matches the schema of the inbound event data.
GeoEvent Definition Name (New) (Conditional)	The name assigned to a new GeoEvent Definition. If a GeoEvent Definition with the specified name already exists, the existing GeoEvent Definition will be used. The first data record received will be used to determine the expected schema of subsequent data records, a new GeoEvent Definition will be created based on that first data record's schema. The parameter is shown when Create GeoEvent Definition is set to Yes and is hidden when set to No.
GeoEvent Definition Name (Existing) (Conditional)	The name of an existing GeoEvent Definition to use when adapting received data to create event data for processing by a GeoEvent Service. The parameter is shown when Create GeoEvent Definition is set to No and is hidden when set to Yes.
Refresh Interval	The period (in seconds) in which a map/feature service feature layer will be polled for feature records. The default is 10 (one poll request will be sent every 10 seconds).
Get Incremental Updates	Specifies whether to request only new or updated feature records using a cached ObjectID or Timestamp value. The default is No. Yes—GeoEvent Server will poll for incremental updates using either a cached ObjectID or Timestamp. No—GeoEvent Server will not incorporate a cached ObjectID or Timestamp in its feature record polls.
Method to Identify Incremental Updates (Conditional)	Specifies how GeoEvent Server will identify newly added or recently updated feature records when polling for incremental updates. Available options include the following: ObjectID—GeoEvent Server will cache the greatest object identifier from the feature record set returned from a map or feature service poll. Only features whose object identifier is greater than the value cached from the last poll will be included in the next poll. Timestamp since last-received to newest-feature timestamp—GeoEvent Server will construct a temporal query with a lower bound and an undefined upper bound. The lower bound will be the date and time of the most current feature obtained from the last poll. The upper bound will not be defined. Only feature records with a timestamp greater than the temporal query's lower bound will be included in the next poll. The lower bound timestamp will be taken from a specified attribute field in the feature record. Remarque : This query may return features with a future timestamp. If this occurs, all subsequent polls will look for data that is greater than the future timestamp. Timestamp interval between last query time until now—GeoEvent Server will construct a temporal query with a lower bound and upper bound. The lower bound will be the date and time the last query was executed. The upper bound will be the date and time now. Only feature records with a timestamp within the temporal query's range will be included in the next poll. The feature record timestamp is taken from a specified attribute field. Timestamp interval between last query time with overlap until now—GeoEvent Server will construct a temporal query with a lower bound equal to a specified number of seconds before the last query was executed and an upper bound equal to the date and time now. Only feature records whose timestamp is within the temporal query's range will be included in the next poll. The parameter is shown when Get Incremental Updates is set to Yes and is hidden when Get Incremental Updates is set to No.
Timestamp overlap duration in seconds (Conditional)	The number of seconds to subtract from the lower bound of the constructed temporal query. Used to artificially expand the temporal range when querying for values between the last query time and now. The parameter is shown when Method to Identify Incremental Updates is set to Timestamp interval between last query time with overlap until now.
Time Field to Identify Incremental Updates (Conditional)	The attribute field whose value will be used when constructing temporal queries. Also used when determining if a feature record's timestamp lies between the lower bound and upper bound of a constructed temporal query. Only attributes whose data type is Date can be selected. The parameter is shown when Method to Identify Incremental Updates is set to a value other than ObjectID.
Query Definition	Specifies a logical expression used to restrict the feature records returned when polling a map/feature service feature layer. The expression is used as the predicate in a Structured Query Language (SQL) WHERE clause. Examples include the following: 1=1—Default, used to construct the clause WHERE 1=1 to poll all available feature records. track_id LIKE '%AJP'—Matches feature records with the literal string AJP anywhere in the TRACK_ID string. reported_dt > timestamp '2019-04-05 22:30:00'—Matches feature records whose date/time attribute value is greater than the specified timestamp.
Use Geometry Filter	Specifies whether a geometry object should be used as a spatial filter to restrict the feature records returned when polling a map/feature service feature layer. A geometry filter, if specified, will be combined with a specified query definition to further restrict feature records returned from the feature layer query. For more information, see Geometry objects. Yes—A spatial filter will be used for querying features from the feature layer. No—A spatial filter will not be used for querying features from the feature layer.
Filter Geometry (Conditional)	The geometry object, expressed using feature JSON, used as a spatial filter. For more information on proper geometry object syntax, see Geometry objects. The parameter is shown when Use Geometry Filter is set to Yes.
Geometry Type (Conditional)	Specifies the type of geometry described by the feature JSON for the geometry filter. Supported geometry types include envelope, point, polyline, polygon, and multipoint. The parameter is shown when Use Geometry Filter is set to Yes.
Spatial Reference (Conditional)	Specifies the well-known ID (WKID) of the spatial reference for the geometry filter. The parameter is shown when Use Geometry Filter is set to Yes.
Spatial Relationship (Conditional)	Specifies the spatial relationship to use when evaluating feature records, polled from a map/feature service feature layer, to determine if they satisfy the conditions of the geometry filter. The parameter is shown when Use Geometry Filter is set to Yes.
Delete Polled Features	Specifies whether the feature records in the record set returned from a feature service should be deleted from the feature service feature layer. The default is No.

Considerations and limitations

The Poll an ArcGIS Server for Features input's unique ability to poll only for newly added or recently updated feature records can be useful when you do not want to ingest, adapt, and process feature records which were previously processed and do not want to delete previously processed feature records from a data source. However, the ability to poll for incremental updates is not resilient when it comes to service restart or server machine reboot. This is because this capability relies on an in memory cache of the latest object identifier or a date and time value to perform successive polling for new or updated records. When either placeholder value is lost after a system restart, the input connector will poll for records that have already been processed previously. In some cases this can create what appears to be false-positive reporting by your GeoEvent Services.

When your solution needs to be more resilient to system restarts, consider implementing the best practice of flagging feature records that are processed with a representative field value. This is recommended instead of using the poll for incremental updates capability. For example, consider adding a field to your feature service's schema called hasBeenProcessed or similar. Have the feature service assign a default value of 0 for new records. When the input connector polls for records, overwrite 0 with a 1 to indicate that the record has been processed. This can be done using a field calculator in your GeoEvent Service. Simultaneously configure your input connector with a query definition WHERE clause of hasBeenProcessed < 1. This guarantees that GeoEvent Server will only retrieve new records whose hasBeenProcessed is 0, meaning new. If you find that you want to reprocess a feature record, ensure the feature editing workflow returns the hasBeenProcessed attribute from 1 back to 0 again so that the records are included in the next poll.

For more information on polling for incremental updates, see the following resources on the Esri Community:

• Poll an ArcGIS Server for Features: Methods used to identify incremental updates
• Polling feature services for "Incremental Updates"
• Question regarding "Incremental Update" workarounds, custom components
• Using a partial GeoEvent Definition to update feature records