Some random ArcObjects that make writing Server Object Extensions easier...(C#)

Server Object Extensions are one of the most powerful features of ArcGIS Server. To make writing them a bit easier for people, I am posting several of the functions I commonly use to deal with incoming data, execute an operation, and provide a response. As an example, I will take the scenario of doing a zonal statistics operation based on a user digitized polygon. The idea is that the user selects a variable for analysis (elevation, temperature, precipitation, etc.), draws a polygon in Flex/JS/Silverlight, the polygon is sent to the server, and zonal statistics for that polygon/variable are returned.


1. JSON to IGeometry: The server will receive a json object representing the polygon digitized by the user. The first step is to convert the JSONObject to IGeometry. I particularly liked NicoGIS's solution to the issue which is shown below:

public IGeometry ConvertAnyJsonGeometry(JsonObject jsonObjectGeometry)
        {
            object[] objArray;

            if (jsonObjectGeometry.TryGetArray("rings", out objArray))
            {
                return Conversion.ToGeometry(jsonObjectGeometry, esriGeometryType.esriGeometryPolygon);
            }

            if (jsonObjectGeometry.TryGetArray("paths", out objArray))
            {
                return Conversion.ToGeometry(jsonObjectGeometry, esriGeometryType.esriGeometryPolyline);
            }

            if (jsonObjectGeometry.TryGetArray("points", out objArray))
            {
                return Conversion.ToGeometry(jsonObjectGeometry, esriGeometryType.esriGeometryMultipoint);
            }

            try
            {
                return Conversion.ToGeometry(jsonObjectGeometry, esriGeometryType.esriGeometryPoint);
            }
            catch
            {
                try
                {
                    return Conversion.ToGeometry(jsonObjectGeometry, esriGeometryType.esriGeometryEnvelope);
                }
                catch
                {
                    return null;
                }
            }
        }  

2. IGeometry to IFeatureClass:  Once I have IGeometry, many times I want to convert it to an IFeatureClass so I can use it in an operation like zonal statistics. Zonal Statistics actually takes an IGeodataset, but IFeatureClass extends IGeodatatset, so all that is required is a cast. Here is a function to convert IGeometry to IFeatureClass:

public IFeatureClass CreateFeatureClassFromGeometry(IGeometry pGeometry, IFeatureWorkspace pOutFeatWorkspace, int wkid = 4236)
        {
            try
            {
                IFields pFields = new Fields() as IFields;
                {
                    // Set up the shape field for the feature class
                    IFieldsEdit pFieldsEdit = (IFieldsEdit)pFields;
                    IField pField = new Field();
                    IFieldEdit pFieldEdit = (IFieldEdit)pField;
                    pFieldEdit.Name_2 = "Shape";
                    pFieldEdit.Type_2 = esriFieldType.esriFieldTypeGeometry;

                    IGeometryDef pGeometryDef = new GeometryDef();
                    IGeometryDefEdit pGeometryDefEdit = (IGeometryDefEdit)pGeometryDef;
                    pGeometryDefEdit.GeometryType_2 = pGeometry.GeometryType;

                    ISpatialReference pSpatialReference;
                    if (wkid == 4326)
                    {
                        ISpatialReferenceFactory2 pSpaRefFact2 = new SpatialReferenceEnvironment() as ISpatialReferenceFactory2;
                        IGeographicCoordinateSystem pGeoCoordSys = pSpaRefFact2.CreateGeographicCoordinateSystem(wkid);
                        pSpatialReference = (ISpatialReference)pGeoCoordSys;
                    }

                    else if (wkid == 102100 || wkid == 3857)
                    {
                        ISpatialReferenceFactory2 pSpaRefFact2 = new SpatialReferenceEnvironment() as ISpatialReferenceFactory2;
                        IProjectedCoordinateSystem pProjCoordSys = pSpaRefFact2.CreateProjectedCoordinateSystem(wkid);
                        pSpatialReference = (ISpatialReference)pProjCoordSys;
                    }

                    else
                    {
                        throw new ArgumentNullException("Invalid Spatial Reference Well Known Id: Please use 4326 for Geographic or 102100 for Web Mercator");
                    }

                    ISpatialReferenceResolution pSpatialReferenceResolution = (ISpatialReferenceResolution)pSpatialReference;
                    pSpatialReferenceResolution.ConstructFromHorizon();

                    pGeometryDefEdit.SpatialReference_2 = pSpatialReference;
                    pFieldEdit.GeometryDef_2 = pGeometryDef;
                    pFieldsEdit.AddField(pField);

                    // Add other required fields to the feature class

                    IObjectClassDescription pObjectClassDescription = new FeatureClassDescription();

                    for (int i = 0; i < pObjectClassDescription.RequiredFields.FieldCount; i++)
                    {
                        pField = pObjectClassDescription.RequiredFields.get_Field(i);
                        if (pFieldsEdit.FindField(pField.Name) == -1)
                            pFieldsEdit.AddField(pField);
                    }
                }

                // Create the feature class
                string sFeatureClassName = "tmp" + Guid.NewGuid().ToString("N");
                IFeatureClass pFeatureClass = pOutFeatWorkspace.CreateFeatureClass(
                   sFeatureClassName, pFields, null, null, esriFeatureType.esriFTSimple, "Shape", null);

                // Add the input geometry to the feature class and return
                IFeatureCursor pFeatureCursor = pFeatureClass.Insert(true);
                IFeatureBuffer pFeatureBuffer = pFeatureClass.CreateFeatureBuffer();

                pFeatureBuffer.Shape = pGeometry;
                pFeatureCursor.InsertFeature(pFeatureBuffer);

                // Flush the feature cursor and release COM objects
                pFeatureCursor.Flush();
                Marshal.ReleaseComObject(pFeatureBuffer);
                Marshal.ReleaseComObject(pFeatureCursor);
                return pFeatureClass;
            }
            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                throw e;
            }
        }

3. Create In-Memory Workspace:  As you notice, the function above requires a workspace.  I like to use in-memory workspaces and to create them I use the following function:

        public IWorkspace CreateInMemoryWorkspace()
        {
            try
            {
                // Create an InMemory workspace factory.
                IWorkspaceFactory workspaceFactory = new InMemoryWorkspaceFactory() as IWorkspaceFactory;

                // Create an InMemory geodatabase.
                IWorkspaceName workspaceName = workspaceFactory.Create("", "MyWorkspace", null, 0);

                // Cast for IName.
                IName name = (IName)workspaceName;

                //Open a reference to the InMemory workspace through the name object.
                IWorkspace workspace = (IWorkspace)name.Open();
                return workspace;
            }

            catch (Exception e)
            {
                Console.WriteLine(e.Message);
                throw e;
            }
        }

4. Get Raster Dataset from Map Service:  Keep in mind that an SOE has access to any of the feature classes in the service which uses the SOE as a capability.  This means that you shouldn't usually need to have any hardcoded data paths in the actual code.  Rather you should try an only use data from the map service itself.  If you send in the index of the value raster the user wants to run analysis on, look how easy it is to open:

public IGeoDataset GetGeoDatasetByMapServiceIndex(IMapServer3 mapServer, int layerID)
        {
            IMapServerDataAccess dataAccess = (IMapServerDataAccess)mapServer;
            return dataAccess.GetDataSource(mapServer.DefaultMapName, layerID) as IGeoDataset;
        }

5. Pre-loading Rasters during Init(): Loading and unloading rasters from memory can be an expensive operation.  Fortunately with SOEs, you can load data into memory when the service starts up, and have it standing by for any request which come in.  This is different than Python geoprocessing services which basically need to load data once the actual request is receive.  One idea I've been using is to create a data dictionary when the service starts up which keeps my value rasters standing by:

public Dictionary<int, IGeoDataset> CreateGeodatasetDictionary(IMapServer3 mapServer)
        {
            Dictionary<int, IGeoDataset> data_dictionary = new Dictionary<int, IGeoDataset>();

            IMapServerInfo msInfo = mapServer.GetServerInfo(mapServer.DefaultMapName);
            IMapLayerInfos layerInfos = msInfo.MapLayerInfos;
            int c = layerInfos.Count;

            for (int i = 0; i < c; i++)
            {
                IGeoDataset data = GetGeoDatasetByMapServiceIndex(mapServer, c);
                data_dictionary.Add(c, data);
            }

            return data_dictionary;
        }

Basically this dictionary is a mapping between map service indexes and their IGeodatasets which I create in the SOE Init() function and store in memory for the life of the SOE.

6. Running Zonal Statistics: Now that we have both the zone dataset and the value raster at our figure tips, the zonal statistics operation is just a couple lines of code.  There are many examples of how to use the zonalops class, but here's the idea:

IZonalOp oZonalOp = new RasterZonalOpClass();
ITable zonalStats = oZonalOp.ZonalStatisticsAsTable(zones, valueRaster, true);
            

7. ITable to IRecordSet: To return the results of the Zonal Stats from the SOE, you must first convert the ITable to an IRecordSet.  Here an straightforward way to do it:

public static IRecordSet ConvertTableToRecordset(ITable table)
        {
            IRecordSetInit recordSetInit = new ESRI.ArcGIS.Geodatabase.RecordSetClass();
            recordSetInit.SetSourceTable(table, new QueryFilterClass());
            IRecordSet recordset = recordSetInit as IRecordSet;
            return recordset;
        }

8. Serialize IRecordset: To convert the IRecordset into return JSON, you can simply use the Conversion.ToJSON method as follows:

IRecordSet return_records = BSharpUtilities.ConvertTableToRecordset(areaTable);
byte[] jsonBytes = Conversion.ToJson(return_records);

return Encoding.UTF8.GetString(jsonBytes, 0, jsonBytes.Length);

That's about it for now.  For information on getting started with SOEs, I would check out ESRI's samples that come with the .NET SDK and also NicoGIS posts 

Great REST-based Server Object Extension posts from NicoGIS

NicoGIS is an awesome GIS blog.  Over the past year, I've been learning REST-based Server Object Extensions for ArcGIS Server 10.  NicoGIS has some one of the best posts on the subject.  I ended up reading the post in Italian for fun, but I bet Google can translate it.

Find Watershed SOE

 The Find Watershed Server Object Extension (SOE) plugs into ArcGIS Server 10 and adds functionality for generating watersheds over REST.  The SOE takes two parameters: hydroshed id and a pour point location.  Proper flow accumulation and flow direction rasters are retrieved by hydroshed id from a file geodatabase.  The location parameter is the pour point.  The goal of the tool is to generate a polygon representing the area of water which flows into a point. 

I like the motto, "if you can't do in on desktop, you won't be able to do it on server."  Keeping this idea in mind, I first wrote the tool as a add-in for ArcMap. This allowed me to debug core business logic for the tool without the complications of the server.

The main difference between a 'desktop tool' and Server Object Extension is the ability to deserialize requests and serialize responses over http.  Since I was new to C# and Server Object Extensions, I first published a sample SOE called SimpleRESTSOE from the samples folder of the ArcGIS 10 install directory. This tool was simply an echo service (i.e. pass a string in, and get the string back).   Requests declare 'text' and 'f' (output format) parameters as part of a url request:

 /echo?text=helloWorld&f=json

and they SOE responds:

{"text":"helloWorld"} 


I got the sample SOE registered with Windows using the command:

RegAsm SimpleRESTSOE.dll /codebase

I then registered the SOE with ArcCatalog and the Server Manager using the companion file located in the samples folder (RegisterSimpleRESTSOE.dll).

Once I verified the sample SOE was running, I simply started modifing the rest schema to take my custom parameters.  To do this, I modified the CreateRestSchema method of the COM class for the tool.  I also added some additional variables to support my business logic:

After modifying the CreateRestSchema method the tool was wired to take the following request:

/echo?hydroshed_id=sa7&location={"x"=-54,"y"=-24}&f=json

I wrapped my business logic in a function and called it from the REST operation's handler EchoInput. THe EchoInput method parses the incoming request, calculates the watershed, and serialize the output.

There is a block of 3 lines towards the bottom that comprise the business logic from my ArcMap tool.  The first two are helper functions (CreateInMemoryWorkspace / CreateFCFromGeometry), and the third is the actual watershed operation:

Global Hydrosheds Lookup

I built a small client application to test Find Watershed over the web. In the app, the user first supplies the location by clicking on the map.  The click location is passes to the server into a query against the lookup map to determine the hydroshed id then :

building a client

Site Selection with ArcObjects

In continuing my application development class through Penn State, I wrote a small site selection tool using C# and ArcObjects. The interesting part of that the application was stand-alone in contrast to the ArcMap tools we wrote in weeks past. I made it through the practice exercises no problem, but ran into some issues when writing the deliverable for the week. Here are some of those problems and solutions:

Referencing Map Layers
One of the bugs in my app which gave me the most trouble was related the order in which layers are loaded into the application. The problem was caused by the following code:

this.axMapControl1.AddLayerFromFile(dataPath + "counties.lyr");
pCntyLayer = this.axMapControl1.get_Layer(0) as IFeatureLayer;
pCntyFC = pCntyLayer.FeatureClass;

this.axMapControl1.AddLayerFromFile(dataPath + "recareas.lyr");
pRecLayer = this.axMapControl1.get_Layer(1) as IFeatureLayer;
pRecFC = pRecLayer.FeatureClass;

this.axMapControl1.AddLayerFromFile(dataPath + "interstates.lyr");
pInterLayer = this.axMapControl1.get_Layer(2) as IFeatureLayer;
pInterFC = pInterLayer.FeatureClass;

this.axMapControl1.AddLayerFromFile(dataPath + "cities.lyr");
pCityLayer = this.axMapControl1.get_Layer(3) as IFeatureLayer;
pCityFC = pCityLayer.FeatureClass;


Notice the index values in the get_Layer function. I initially expected the layer indexes to be the same as the order in which they were added to the map. This turned out to be wrong. Actually each layer was added to the beginning of the array, meaning that for the code above, each of the indexes should be 0.

This bug through me on a wild goose chance because it turned out I got the first layer index correct, so I didn't get a error message until I was further into writing application logic. In the code above, each of the layers will be the county layer. By coincidence this was the first layer I was doing operations one which cause additional confusion when operations on the cities layer didn't work.

Creating the Geometry Bag
When creating a geometry bag of features from Query Filter, I first attempted writting my own functions but quick found that the professors solution was more elegant. I translated the VBA code into the following C# function:


private IGeometryBag getGeomBag(IQueryFilter pQueryFilter, IFeatureClass pFClass)
{
IEnumGeometry pEnumGeom = new EnumFeatureGeometryClass();
IEnumGeometryBind pEnumGeomBind = pEnumGeom as IEnumGeometryBind;
pEnumGeomBind.BindGeometrySource(pQueryFilter, pFClass);
IGeometryFactory pGeomFactory = new GeometryEnvironmentClass();
return pGeomFactory.CreateGeometryFromEnumerator(pEnumGeom) as IGeometryBag;
}


This function applies the filter and return a nice geometry for additional site selection steps.

Conclusion
While I got my stand alone application working, I was unable to integrate it into excel and powerpoint. I am currently experimenting with VSTO (Visual Studio Tools for Office) to see if I find a solution.

A Friendly Way to Deliver Custom ArcMap Tools

After experimenting with different ways to distribute a custom ArcMap tool, the most friendly for the end-user seemed to be an ArcGIS extension delivered with an installer package.  The following will go through some basic steps for coding an ArcMap extension/installer in C# 2008, and assumes you have already developed a custom COM based menu or toolbar. In my case, I am using a simple tool which adds ortho-rectified imagery to an ArcMap session based on the location of a user's click (although the code for the tool will not be discussed).

Coding an ArcMap Extension

Anybody who has used tools from Spatial Analyst or 3D Analyst is familiar with ArcMap extensions.  To enable ArcMap extension, you simply open Tools --> Extensions, and select the appropriate checkbox.

Once you have a custom tool or toolbar written and successfully tested, it is very easy to wrap the tool(s) up as an extension just like Spatial Analyst. Let's take a look.

The first thing you want to do is create a new COM class in the same Visual Studio project as your tools.  Your new class will need to implement the IExtension and IExtensionConfig Interfaces.   Make sure that you include the necessary imports.  Notice the two module level properties m_app and m_extensionState. The first will be used during the implementation of IExtension, and the second for IExtensionConfig.

Looking at the implementation of the IExtension interface below, we see the m_app property which is set during the IExtension's startup method and holds the hook into the ArcMap instance. Here I have also included a name for the extension, but did not add any additional logic to be executed during shutdown.

  With IExtension finished, we move on to IExtensionConfig setting the return message for the extension's description and coding the State property which sets and return the m_extensionState variable.  The State method store info on which the extension has been enabled/disabled from the ArcMap tools -> extensions dialog.

 To round out our extension class, we add the component category registration which registers the component with the Windows registry.  As you would guess, MxExtension is used instead of MxCommandBars or MxCommands.

To finish development of the extension, I have added an additional method to a utilities class which I use to store functions which will be shared among tools.  This new function returns a boolean which indicates whether or not the extension is currently enabled. In this project, I also wanted to ensure layers were present in the map document which is why there is some additional logic.  It would probably be better to separate that logic out into its own function to increase reuse-ability for future endeavors.

I then use this function inside the Enabled method of my tool to add the additional prerequisite that the extension be enabled.

That wraps up creating the extension.  To review, we created a new extension class, implemented the IExtension, and IExtensionConfig interfaces, and rigged up the logic to the ortho tool to enable only when the extension has been enabled.

Creating an Installer/Setup file

Now that we have an extension for the ortho-tool, it is time to create the installer class which will sit inside the same Visual Studio project.  To do this just add a new item and select the Installer Class template.  Switch to the code view and override the methods shown below.  The Install/Uninstall methods will now handle registering/unregistering our tools Assembly with Windows.

 Next, we create a brand New Project, select Other Project Types, and Setup and Deployment Project.  My tools were part of a project called "Boston Tools" so I call the setup project "BostonToolsSetup".  To link the two project together creating our friendly solution for distributing the tools:

• Click on the new project and customize the manufacturer and product name properties for the tool.  These properties will also be used forming the path for where the extension is install on the users computer.
• Right-Click the setup project and select Add->Project Output.  You can then select Primary Output and the ArcMap Extension's project name. 
• Once you see the dependencies are added to the setup project, right-click the setup project again and select view->custom actions.  Double-click on the Application Folder and confirm that Primary Output from is selected and click OK. 

Now you can build the setup package and the setup.exe / project.msi for friendly distribution of your tool, but note that when you make changes to your tool, you will also need to rebuild the setup project.


 

Developing COM Components to Plug Into ArcMap - My Noobie Problems

After my initial experience writing the War card game in VB and C#, I've decided to try and stick with C# for the remainder of GEOG 489. The problem with this is that while the IDE is the same, I've quickly found differences in working with ArcObjects in C# vs. VB.

No COM Class Template

The first major difference I found was that C# does not have a built-in COM Class template. Duh! Luckily within about 10 minutes I found one at Oren Ellenbogen's blog which was written for C# 2005, but it seems to be working just fine for 2008. I place the zip file in 'Visual Studio 2008\Templates\ItemTemplates\Visual C#' and it worked like a charm.

No Stub

The next issue that I ran into was that for some reason C# was not generating the stub code for implementing the ICommand interface. I had noticed this on the last project, but had simply brute forced it because there were relatively few methods to implement. This time I learned that if you right-click on the window, you can tell the IDE to stub out the methods:


In my opinion, if you have to write out all the stub code then you might as well be working in notepad.

ArcMap Don't See My Component

After writing the code to create the custom zoom button, I had a more frightening problem of not seeing the command category appear in ArcMap. I was worried that this might take a while to troubleshoot, but I quickly discovered a couple of issues that may have been the source of my problem. The key to solving them was finding a long tread on the bytes.com forums discussing a similar problem. I believe that the key to solving my problem was:

• Adding a blank constructor to the class (seemed pretty logical)
• Going to Project -> Properties "Build" tab and selecting the "Register for COM Interop" option

These steps resolved my problem. I was then able to see the "Geog 489" category in ArcMap and the zoom tool worked with no problem.

No Modules in C#

When starting section III of the project, I was surprised that there was no option to add a "module" in the add item dialog. I soon realized that the idea of Modules is not part of C#, instead everything must be in its own class. In figuring out the best achieve a similar solution for re-using code (DRY...don't repeat yourself...or even better DIE...duplication is evil), it stumbled upon a forum thread discussing this exact issue. The post recommends creating a public sealed class. I found this worked well and I made the Zoom method static so I could call the function without actually instantiating the class. Here's what the class looks like: