Python Geoprocessing Vs. C# Server Object Extensions (AGS 10)

The following deals with development of geoprocessing services specifically for ArcGIS Server 10

Introduction Video I made for Penn State's MGIS Program

Over the past weeks, I have written several server-side tools to support web mapping applications as part of my capstone project for Penn State's MGIS Program.  The following is a general overview of what I've found to be the strengths and weaknesses of  Geoprocessing Services in Python and Server Object Extensions in C#.

Over the coming weeks, I will be releasing the code and discussions of the individual tools in separate posts. 

Geoprocessing Services (Python)

For developers of web-based geoprocessing tools for ArcGIS Server 10,  the arcpy Python library is now a popular alternative to the more complex ArcObjects library.  With arcpy, you can access a decent percent of the ArcGIS tools with a clear syntax for linking them together. Knowledge of object-oriented programming is not required, and you can integrate GIS with an entire world of Python third-party libraries.  To web-enable arcpy, scripts are added to toolboxes which get published to the server.

Python Pros

• Accelerated development time
• ArcGIS includes many Python code samples
• Easy to debug business logic

In spite of their ease of development and clear syntax, aspects of python geoprocessing services limit their power.  First is that they execute slower that the equivalent code written in .NET ArcObjects. For desktop tools that may not be a problem, but for the web this can negatively impacts user experience. 

Even if speed was not an issue, geoprocessing services are more cumbersome to move from one environment to another (e.g. staging -> production) because access to required source data is more tightly coupled with the tool.  Take the example of a server-side point clusterer.  To access the point feature class containing un-clustered points, a python script may use either of the following logic:

This first example uses a hard-coded absolute path that will need to be changed if the tool moves to an environment with a different directory structure.  A better choice would be the relative path example, but there is still hard-coding of the dataset's geodatabase and name.  With either choice, the tool suffers from a lack of true encapsulation which ends up making it harder to move and reuse on other point layers.

An aspect of the ArcGIS workflow for creating scripting tools which hinders a tool's flexibility is the need for explicit registration of input and output parameters in a properties dialog.

Python Cons

• Slower execution speed when compared to C#
• Tighter coupling with source data
• Double managing of service input/output parameters

When using geoprocessing services, the developer gains development time by sacrificing execution speed, and the fine-grain control offered by ArcObjects.

C# Server Object Extensions (ArcObjects 10 .NET SDK)

C# Pros

• Faster code execution
• Loose coupling with tool source data
• Easy to deploy across environments

I began using .NET ArcObjects because python geoprocessing service felt sluggish.  The allure of greater speed justified learning a more complex language.  What I found was not only increased speed, but also easier deployment and lower maintenance.

The idea of the Server Object Extension is that instead of publishing a service on top of the server, you extend the server itself.  This extended functionality can then be enabled for any service along with other out-of-the-box capabilities (e.g. Export Map, Identify, Find, Query, Generate KML, WMS).

The beautiful thing about Server Object Extensions is that ArcObjects has hooks into the source data within a map service.  These hooks give C# access to datasets using map service layer indexes instead of directory paths allowing for looser coupling with the server environment. ESRI Web APIs also use map service layer indexes which makes client-server communication easier.










C# Cons

• Large and complex library
• Not Cross-Platform (won't run on AGS Java version).
• Steeper learning curve

Writing SOEs requires knowledge of a more complex library of tools accessed through a more complex programming language.  If a project does not have somebody with ArcObjects experience, diving into Server Object Extensions will be intimidating. In ArcObjects, finding the core methods for geoprocessing is easy, but getting data into a class which implements the correct interface for use as input in those methods can be difficult.

An example of this would be getting a geometry object, passed in as JSON from the client, into a feature class which can be used by a Spatial Analyst tool.  The solution I used took 80 lines of not fun code to write.  The image below is meant to give an impression of the complexity.

Another area of trouble is .NET and inability to run outside of a Window's environment.  Being that ESRI offer's a Java version of the ArcGIS Server, choosing C# means limiting a tool to Window's and the .NET version of ArcGIS Server.  This hasn't been a major issue for me yet, but I want as many options in hosting environments as possible.

Overall I have found the benefits of Server Object Extensions outweigh the ease of development offered by  Python when performance and reusablilty are important. Initially, I found the complexity of working in ArcObjects discouraging, but slowly I wrote function to deal this data type conversions and development got easier.  I now consider arcpy as a solution only on the desktop.

Exception Handling in ArcGIS Python

When writing python scripts, it is very important to add some basic exception handling so you're not pulling your hair out when a script doesn't work.

Recently I was running a geometry simplification script I've been using for about two years. I had never had a problem with it, but suddenly I started ERROR 999999: Error executing function. This error is basically ArcGIS saying "I have no idea what happened". Man I was bummmin...no lines numbers...no idea why after so long I had a problem. Was it the data? Well, yes...it turned out the Shape field was named 'Shape_' instead of 'Shape'.

To find the error, I wrapped the script inside a try/except block and used two functions in my helper class:


def reportGPErrors(self, geoprocessor):
    '''
    reports any geoprocessing related errors
    '''
    gpmsgs = geoprocessor.GetMessage(0)
    gpmsgs += geoprocessor.GetMessages(2)
    geoprocessor.AddError(gpmsgs)
    print gpmsgs

def reportSysErrors(self, geoprocessor):
    '''
    reports python system related errors
    '''
    tb = sys.exc_info()[2]
    tbinfo = traceback.format_tb(tb)[0]
    pymsg = "PYTHON ERRORS:\nTraceback Info:\n"
    pymsg += tbinfo + "\n"
    pymsg += "Error Info:\n" + str(sys.exc_type) + ": "
    pymsg += str(sys.exc_value) + "\n"
    geoprocessor.AddError(pymsg)
    print pymsg


Both of the functions take a geoprocessor, which gives the ability to print to both the python console and the ArcGIS message window.

To implement these function in geoprocessing scripts, simply import helpers class, and place a call to these function in the try/except block:


from helpers import Helpers
oHelpers = Helpers()
gp = oHelpers.createGeoprocessor()

try:

    code...code...code

except:
    oHelpers.reportGPErrors(gp)
    oHelpers.reportSysErrors(gp)


Creating the Python Geoprocessor

I was recently hired to refactor about 20 geoprocessing services written in python. At the beginning of each script was about 5 lines of boiler plate to instantiate the geoprocessing object (gp), check out the appropriate extensions, and set the necessary environmental variables. Each script began with more or less:


import arcgisscripting
gp = arcgisscripting.create(9.3)
gp.overwriteoutput = 1
gp.CheckOutExtension("Spatial")
gp.cellSize = "0.0041666667"


Since this code was being repeated throughout all of the gp services, I simply turned it into a function in my helpers class. This removed about 100 lines of boiler plate and gave the ability to fine-tune the geoprocessor for all of the services. There are some additional debugging methods inside createGeoprocessor which also reside in the helpers class. I will do a future post on those basic error handling functions soon.


def createGeoprocessor(self):
    '''
    Returns a 9.3/9.3.1 Geoprocessor whose
    overwrite property is set to true
    and Spatial Analyst extension checked-out
    '''
    import arcgisscripting
    gp = arcgisscripting.create(9.3)
    gp.overwriteoutput = 1
    try:
        if(gp.CheckExtension("Spatial") == "Available"):
        gp.CheckOutExtension("Spatial")
    else:
        raise "LicenseError"
    except "LicenseError":
        self.toScreen(gp, Messages.SPATIAL_ANALYST_ERROR)
    except:
        self.reportGPErrors(gp)
        self.reportSysErrors(gp)
    return gp