Warning: An exception occurred in FDO component.

I got the following warning in MapGuide log file:

Warning: An exception occurred in FDO component.
        Zeichenfolge ist kein gültiger Ausdruck. 
 StackTrace:
  - MgStylizationUtil.ExceptionTrap() line 231 file SE_ExpressionBase.cpp

 
The message means "string is invalid expression".

In my case it seems the issue is related to text label style. The text label was originally published using MTEXT as style. Later on - due to a bug in MapGuide - I changed the text style to TEXT. But since I changed TEXT back to MTEXT the message doesn't appear in the log files anymore. 


Autodesk Infrastructure MapServer 2017

Uploading MapGuide package - Exception

Uploading MapGuide package - Exception

If you modify a MapGuide package make sure that you zip the correct files and folders afterwards.
I unzipped a MapGuide package, changed some values and zipped the folder but received the following error message when trying to upload the file:


Maestro error message:

value cannot be nul
Parametername: entry

System.ArgumentNullException:  value cannot be nul
Parametername: entry
   bei ICSharpCode.SharpZipLib.Zip.ZipFile.GetInputStream(ZipEntry entry)
   bei Maestro.Packaging.PackageBuilder.UploadPackageNonTransactional(String sourceFile, UploadPackageResult result) in C:\working\JenkinsCI\home\slave_win\jobs\Maestro trunk\workspace\Maestro.Packaging\PackageBuilder.cs:Zeile 217.
   
MapGuide error message:

   ERROR_MESSAGE:    An exception occurred in DWF component. File not found in archive
STACK_TRACE:    - MgLibraryRepositoryManager.LoadResourcePackage() line 183 file c:\working\build_area\mapguide\2.5.2\x64\mgdev\server\src\services\resource\LibraryRepositoryManager.cpp - MgResourcePackageLoader.Start() line 150 file c:\working\build_area\mapguide\2.5.2\x64\mgdev\server\src\services\resource\ResourcePackageLoader.cpp - MgResourcePackageLoader.CreateByteReader() line 108 file c:\working\build_area\mapguide\2.5.2\x64\mgdev\server\src\services\resource\ResourcePackageLoader.cpp - MgZipFileReader.ExtractArchive() line 61 file c:\working\build_area\mapguide\2.5.2\x64\mgdev\server\src\services\resource\ZipFileReader.cpp


My mistake was to zip the top level folder - which I got when I unzipped the package. But you need to zip the content of the top level folder, not the top level folder itself. 

AIMS 2017

MapGuide Logfile - two warnings and what they mean

MapGuide log file contains the following warnings:

<2017-02-14T08:54:36> 4712 Ajax Viewer 10.99.66.2 Administrator
 Warning: An exception occurred in FDO component.
        Beim Abrufen eines Eigenschaftenwerts stimmte der Eigenschaftentyp nicht überein. 
 StackTrace:
  - MgStylizationUtil.ExceptionTrap() line 813 file StylizationEngine.cpp
  
the error message translates roughly as 
  "property value and property typ do not match"
  
The message was caused by the following filter definition:  

W_ID_TYPE  = 1 AND  (   W_ID_TYPE_IP  NULL    OR   NOT  W_ID_TYPE_IP  IN ( '1') )

W_ID_TYPE_IP is numeric but the value is given as string.  
  
 The second one:
  
<2017-02-14T12:19:44> 3444 10.99.66.65 Anonymous
 Warning: An exception occurred in FDO component.
        Die Kennung 'w_id_type' wurde nicht erkannt. 
 StackTrace:
  - MgStylizationUtil.ExceptionTrap() line 813 file StylizationEngine.cpp  
  
the error message translates roughly as 
  "property 'w_id_type' not found."

The message was caused by the following filter definition:  

w_id_type  IN ( 11)   AND NOT  ID_SITE_TYPE  IN (2007)

Property names as case-sensitive. The feature class does not contain 'w_id_type' but 'W_ID_TYPE'.


Unfortunately for both warnings no further information is given to where exactly the issues occur. I stumbled across the cause of the first warning by looking for the cause of the second warning. The second warning tells you exactly what the problem is - unfortunately I only realized that after I had spent several hours looking for it. 'w_id_type' is an attribute we use in many feature classes and 71 layers reference the attribute in filter expressions. It was somehow obvious that there is a problem in one of the layers but not in all of them as the warning did not appear all the time. Took me quite some time to figure out what was going on - despite the warning being clear about it.

AIMS 2017





  
  

MapGuide - renderMap - transparency

The RenderMap functions takes a MgColor object in order to set the background color of the map.
I wanted to have a transparent background - according to the API docs you can create a MgColor object by providing values for RGB and alpha:

MgColor(int red, int green, int blue, int alpha);

Problem is, the map background wont show up transparent. In order to have the background transparent MgColor needs to be called by passing hexvalues instead:

MgColor("ffffff00");

AIMS 2017

AIMS 2017 - new error message

After updating from AIMS 2013 to AIMS 2017 no raster images were shown. Any preview in Studio triggered the following error message in the log file:


<2016-09-07T08:33:35> 3752 Ajax Viewer 10.99.66.114 Administrator
Error: Stilisierung von folgendem Layer fehlgeschlagen: Luftbild_1pro
Ausnahmefehler in FDO-Komponente aufgetreten.
Ausnahmefehler in FDO-Komponente aufgetreten.
'FdoATILSession::GetImage': Funktion kann aufgrund eines ungültigen Wertes für Eingabeparameter 'fileDescriptor' nicht ausgeführt werden. (Ursache: , grundlegende Ursache: 'FdoATILSession::GetImage': Funktion kann aufgrund eines ungültigen Wertes für Eingabeparameter 'fileDescriptor' nicht ausgeführt werden. )
StackTrace:
- MgMappingUtil.StylizeLayers() Zeile 899 Datei e:\build\2017\re_ims2017_46_1-6376417\ent\os\server\src\services\mapping\MappingUtil.cpp  


This message was new to me and hadn't occurred with AIMS 2013.

Also, "Testing connection" always returned "OK" for our raster file data source. The raster file data source uses an ALIAS which points to a shared network drive.

Problem was, that AIMS 2017 was running under local user account. In order to access the network drive AIMS service needs to run under different user account. 

AIMS 2017, SP1

AIMS GeoRest - Update 2013 to 2017

I installed AIMS 2017 on our test server. To get Georest extension running again the following steps were required:

(1) change path for virtual folder "rest" to ..Autodesk\Autodesk Infrastructure Web Server Extension 2017\www\GeoREST\bin
(2) re-create Wildcard Module Mapping for GeoRest_ISAPI.dll
(3) copy all GeoRest configuration files from old installation into ..Autodesk Infrastructure Web Server Extension 2017\www\GeoREST\conf
(4) change Port number in conf files (old port number: 2801, new port number: 2811)

Restart IIS, test, done.

AIMS 2017,

Why you should restart AIMS/MapGuide when testing ....

I noticed by chance that a few layers - which are all based on one Oracle view - were significantly slower in our production system but no performance issue occurred in our test environment. Whilst investigating the issue I made a few mistakes which made the whole process of finding and fixing the issue more time consuming than necessary. Here are the details:

After noticing the issue I asked my colleagues if they had any idea why this is and whether anything had changed recently. None of them had an explanation so I spent some time checking / testing:

- running performance reports in MapGuide/AIMS noticing that time spent on "Layer" is 20times slower in production
- comparing view definitions but only cursorily (looking at WHERE clause only) : mistake #1
- re-creating spatial index for base table : no change
- running 1-click maintenance in Map-Administrator : no change
- copy a layer definition from production to test environment : no change
- trying to load view into other applications to see whether the issue is related to application or to view but GeoRaptor couldn't add layer to map due to rowid issue, QGIS rejected view complaining that view didn't have a proper spatial index and TatukGIS Viewer basically saying something similar (first the message about the spatial index issue made me re-re-create the spatial index but testing further views in QGIS/TatukGISViewer I got the same messages for all other views as well).
- running a few spatial SQL queries in SQLDeveloper but there wasn't any significant difference between test and production : mistake # 2 (this result caused quite some confusion as I did not have an explanation, it did not fit and even contradicted other findings!)

After spending some time on running these tests a colleague came back to me and told me, that the view had changed recently but when he applied the change he didn't notice any performance issue whilst generating graphics in Map. The change was made by adding a subquery to the SELECT clause. He asked if the change should be removed but I replied that I will do some tests now taking the new information into account : mistake # 3.

As the change to the view definition had only been applied in the production system I applied the change in the test system as well - here mistake # 4 occurred. I then ran the MapGuide/AIMS performance report again but the performance had not decreased. Performance was good as before. Then I changed the data source of the Oracle user I was testing with and pointed it to our production database. When I ran the test again performance was slow. I stopped any further testing for the day and did something else.

At this point my theory was that the same query has different execution plans in Oracle (production and test) therefore different execution times due to different Oracle settings. What didn't really fit was that when I ran the spatial queries in SQLDeveloper I didn't see any significant difference in performance. But I didn't have anything else to test so finally I decided to replace the view definition in production with the previous view definition - basically removing the change which had been applied recently. I re-ran the performance report in MapGuide/AIMS and suddenly I got an error message:

Failed to stylize layer: DP_xxx
An exception occurred in FDO component.
ORA-00904: "G"."OWNER_OBJTYP": invalid identifier
...

Now I realised one of my mistakes. MapGuide/AIMS caches certain settings/definitions. I don't know how it works in detail but probably roughly like that:
- when AIMS/MapGuide starts it reads the Oracle schema and caches some information
- (SELECT) statements are being build based on the cached information and used against Oracle

For a view MapGuide/AIMS seems to read the view definition and then creates a SELECT statement which includes all columns of the view. 
If the view definition in Oracle changes and new columns are added to the view MapGuide can still run the SELECT statement against the new view definition - basically just ignoring all newly added columns.  But when columns have been removed from the view the SELECT statement still includes those columns and Oracle will response with ora-00904.

Mistake 4 - when I applied the change to view definition in the test system I did not restart MapGuide/AIMS. MapGuide was still using the SELECT statement which did not include the newly added column in the view. But exactly that new column had a negative impact on performance.
If I had restarted MapGuide/AIMS at this point in time I could have avoided further testing and head-scratching.

Mistake 3 - if I had listened to my colleague who offered to remove the change in production I might have noticed the cause of the problem earlier (because even without restarting MapGuide/AIMS we would have seen the Ora-00904 message in the log file).

Mistake 2 - my spatial query was simplified, the select clause included only "fid, geom". As it did not include the column which caused the performance issue there is really no surprise that performance in both production and test were nearly the same. Conclusion: always use (if possible) exactly the same (SELECT) statement as the application when you do testing!

Mistake 1 - I should have noticed the difference in view definitions myself just by copying the view definitions from production and test into an editor and then noticing the different numbers of lines....

So at the end a trivial issue but it took longer then necessary to get it fixed.

Why you should restart AIMS/MapGuide when testing ....

I noticed by chance that a few layers - which are all based on one Oracle view - were significantly slower in our production system but no performance issue occurred in our test environment. Whilst investigating the issue I made a few mistakes which made the whole process of finding and fixing the issue more time consuming than necessary. Here are the details:

After noticing the issue I asked my colleagues if they had any idea why this is and whether anything had changed recently. None of them had an explanation so I spent some time checking / testing:

- running performance reports in MapGuide/AIMS noticing that time spent on "Layer" is 20times slower in production
- comparing view definitions but only cursorily (looking at WHERE clause only) : mistake #1
- re-creating spatial index for base table : no change
- running 1-click maintenance in Map-Administrator : no change
- copy a layer definition from production to test environment : no change
- trying to load view into other applications to see whether the issue is related to application or to view but GeoRaptor couldn't add layer to map due to rowid issue, QGIS rejected view complaining that view didn't have a proper spatial index and TatukGIS Viewer basically saying something similar (first the message about the spatial index issue made me re-re-create the spatial index but testing further views in QGIS/TatukGISViewer I got the same messages for all other views as well).
- running a few spatial SQL queries in SQLDeveloper but there wasn't any significant difference between test and production : mistake # 2

After spending some time on running these tests a colleague came back to me and told me, that the view had changed recently but when he applied the change he didn't notice any performance issue whilst generating graphics in Map. The change was made by adding a subquery to the SELECT clause. He asked if the change should be removed but I replied that I will do some tests now taking the new information into account : mistake # 3.

As the change to the view definition had only been applied in the production system I applied the change in the test system as well - here mistake # 4 occurred. I then ran the MapGuide/AIMS performance report again but the performance had not decreased. Performance was good as before. Then I changed the data source of the Oracle user I was testing with and pointed it to our production database. When I ran the test again performance was slow. I stopped any further testing for the day and did something else.

At this point my theory was that the same query has different execution plans in Oracle (production and test) therefore different execution times due to different Oracle settings. What didn't really fit was that when I ran the spatial queries in SQLDeveloper I didn't see any significant difference in performance. But I didn't have anything else to test so finally I decided to replace the view definition in production with the previous view definition - basically removing the change which had been applied recently. I re-ran the performance report in MapGuide/AIMS and suddenly I got an error message:

Failed to stylize layer: DP_xxx
An exception occurred in FDO component.
ORA-00904: "G"."OWNER_OBJTYP": invalid identifier
...

Now I realised one of my mistakes. MapGuide/AIMS caches certain settings/definitions. I don't know how it works in detail but probably roughly like that:
- when AIMS/MapGuide starts it reads the Oracle schema and caches some information
- (SELECT) statements are being build based on the cached information and used against Oracle

For a view MapGuide/AIMS seems to read the view definition and then creates a SELECT statement which includes all columns of the view. 
If the view definition in Oracle changes and new columns are added to the view MapGuide can still run the SELECT statement against the new view definition - basically just ignoring all newly added columns.  But when columns have been removed from the view the SELECT statement still includes those columns and Oracle will response with ora-00904.

Mistake 4 - when I applied the change to view definition in the test system I did not restart MapGuide/AIMS. MapGuide was still using the SELECT statement which did not include the newly added column in the view. But exactly that new column had a negative impact on performance.
If I had restarted MapGuide/AIMS at this point in time I could have avoided further testing and head-scratching.

Mistake 3 - if I had listened to my colleague who offered to remove the change in production I might have noticed the cause of the problem earlier (because even without restarting MapGuide/AIMS we would have seen the Ora-00904 message in the log file).

Mistake 2 - my spatial query was simplified, the select clause included only "fid, geom". As it did not include the column which caused the performance issue there is really no surprise that performance in both production and test were nearly the same. Conclusion: always use (if possible) exactly the same (SELECT) statement as the application when you do testing!

Mistake 1 - I should have noticed the difference in view definitions myself just by copying the view definitions from production and test into an editor and then noticing the different numbers of lines....

So at the end a trivial issue but it took longer then necessary to get it fixed.

Publish to MapGuide/AIMS - Spatial Filter II

As described in a previous posting  when you publish data from Map to MapGuide/AIMS the published layer files get a spatial filter assigned. The filter will decrease performance and needs to be removed. So usually we remove the spatial filters after publishing from Map.

But today we had user reporting to us that some layers don't appear on the map although they used to appear in the past.

As it turned out spatial filters had been assigned to those layers but the filters covered our area of interest and therefore no one noticed it. But a few weeks ago we switched to a new coordinate system and had to transform our spatial data into the new system. The spatial filters defined in some layers had not been transformed as we had not been aware of them and assuming all had been removed. After removing the spatial filters the layers re-appeared on our maps.

The issue with regards to layers getting a spatial filter assigned when being published has been reported to Autodesk some time back. Maybe it has been fixed by now in newer releases.

Here is the core bit of the PHP based script for removing spatial filters - as usual a quick and dirty solution:

 if ($_POST['hiddenparam'] == '') die();       
      if ($_POST['projektpfad'] == '') die();       
      $MGAdminPassword = $_POST['password'];  
      try  
      {                      
           include './config.schema.list.php';  
           include '../vaw.lib/initializeMapGuide.php';  
           $Verzeichnis = $_POST['projektpfad'];             
           echo "<html><body>";  
           echo "<p>MapGuide Pfad: $Verzeichnis </p>";  
           $VerzeichnisResourceId = new MgResourceIdentifier($Verzeichnis);  
           $LayerReader = $resourceService->EnumerateResources($VerzeichnisResourceId, -1, "");  
           $Xml = $LayerReader->ToString();  
           $LDomDoc = DOMDocument::loadXML($Xml);  
           $pfad = new DOMXPath($LDomDoc);  
           $frage = "//ResourceDocument/ResourceId";  
           $ebenen = $pfad->query($frage);  
           $anzlayer = 0;  
      foreach($ebenen as $knoten)  
      {  
           $Layer = $knoten->nodeValue;  
           $LayerResourceId = new MgResourceIdentifier($Layer);  
           $layerName = $LayerResourceId->GetName();  
           $layerType = $LayerResourceId->GetResourceType();  
           if($layerType == 'LayerDefinition')  
           {  
                echo "<b>Layername: $layerName </b><br>";  
                $tmpReader = $resourceService->GetResourceContent($LayerResourceId);  
                $layerXML = $tmpReader->ToString();                 
                $doc = DOMDocument::loadXML($layerXML);                            
                $FilterNode = $doc->getElementsByTagName('Filter');  
                foreach($FilterNode as $node)   
                {  
                     if(!stristr($node->nodeValue, "GeomFromText"))  
                     {                      
                          //kein Spatial Filter  
                          //echo " <b>Kein Spatial Filter.</b><br>";  
                     }  
                     else  
                     {  
                          $node->nodeValue = '';  
                          echo "<b>Spatial Filter - vorhanden und entfernt.</b> <br><br>";  
                          echo "<small>vorhandene Filter: {$node->nodeValue} .</small>";  
                     }  
                }                           
                //echo $doc->ToString();  
                $modifiedLayerDefinition = $doc->saveXML();  
                $byteSource = new MgByteSource($modifiedLayerDefinition, strlen($modifiedLayerDefinition));  
                $byteSource->SetMimeType(MgMimeType::Xml);  
                //$resourceId = new MgResourceIdentifier($LayerResourceId);  
                $resourceService->SetResource($LayerResourceId, $byteSource->GetReader(), null);  
           }  
      }  
 }  
 catch ( MgException $e )  
   {  
     $errorMsg = $errAuthenticationFailed;  
   }  
   catch ( Exception $e )  
   {  
     $errorMsg = $e->getMessage();  
   }  

Map 2013, SP2

MsMpEng.exe Slowing down MapGuide

The blog title is not entirely correct - MapGuide/AIMS is not directly affected, but - here are the details:

Suddenly (Monday morning) users started to complain that our AIMS based WebGIS is particularly slow. As it turned out it was not AIMS server itself but the application around it which was slow. Opening a form  or performing a search suddenly took seconds,  but map creation was still fast as usual. We do not use AIMS's Industry Model but TBVIEW1  as an better performing alternative. 

We have three servers in use - the issue occurred only on one of them. TaskManager showed some unusual activity on the machine affected - it appeared as process MsMpEng.Exe was running nearly constantly consuming 25% of CPU power. Further investigation showed, that MsMpEng got active as soon as the WebGIS application interacted with Oracle, for instance when opening a form and remained active for some time. No other processes were using CPU power at the same time. For testing purpose our IT disabled MsMpEng.exe temporarily and the performance returned back to normal. They are now investigating why this happens and why it happens only on one of three machines (W2008R2). 

AIMS 2013, SP2

MapGuide - Server Admin

MapGuide / AIMS mapagent web site contains a section called "Server Admin". If you try to execute one of the commands you will probably get an error message:

MapGuide Server Admin Service Tests
Problem:\nCannot establish connection to the Infrastructure Map Server.\n\nPossible Causes:\n-Your session has been idle for too long;\n-Infrastructure Map Server is no longer responding;\n-Internet connection problems.\n\nSolution:\nRestart your application / session, restart the Infrastructure Map Server service, or contact with the server administrator.Problem:\nCannot establish connection to the Infrastructure Map Server.\n\nPossible Causes:\n-Your session has been idle for too long;\n-Infrastructure Map Server is no longer responding;\n-Internet connection problems.\n\nSolution:\nRestart your application / session, restart the Infrastructure Map Server service, or contact with the server administrator.


If so then change "localhost" to "127.0.0.1" in file "serveradminhelper". The file can be found in the mapviewer sub folder. Depending on your installation your viewer is either .net, Java or PHP based. For a .net based view the path is:

\www\mapviewernet\

The line in question is (für ASPX):

 serverAdmin.Open("localhost", cred);
and needs to be changed to
 serverAdmin.Open("127.0.0.1", cred);


AIMS 2013.

dynamic feature labels not always displayed

We noticed that dynamic labels for FDO features are not always displayed. Depending on other map features (points and text) dynamic labels are generated or not. Here is one example: the blue text is a dynamically generated label for a grid cell:

dynamic label in centre of map.....

If the map extent is slightly different or additional layers are turned on the dynamic label disappears:

...disappears when other layers become visible

Unfortunately there is no "switch" to force AIMS to generate the label nonetheless. But Autodesk support kindly provided a more detailed explanation about what is going on and how to get around it:

"AIMS/Map: labels are not visible if point feature or other labels are at the same place
You miss labels coming from FDO Feature stylisation
 
Cause 
In general labels are visible always on top of the graphical objects, but labels are not visible if point features are at the same place
  
Solution 
As a workaround you can change the following settings in the .layer file of Map 3D or in the .xml of AIMS
• Save the FDO layer for point features class as .layer file
• Open the .layer file with notepad(++)
• Change following setting in the .layer file from True to False
                    <AddToExclusionRegion>false</AddToExclusionRegion>"

Map 2013, SP2

MapGuide / AIMS performance

Most of the data we publish to our intranet web gis (based on AIMS 2013) comes from Oracle via FDO Topobase provider. For performance reasons we have created an SDF file for the most often used base data - containing 30+ tables and 700'000 features. To my surprise the SDF file is not as fast as I expected. By changing the default value "DataConnectionPoolExcludedProviders" performance increased noticable.

By default SDF files are excldued from ConnectionPooling:
 
DataConnectionPoolExcludedProviders = OSGeo.SDF,OSGeo.SHP

By uncommenting the line and restarting AIMS things got better:

#DataConnectionPoolExcludedProviders = OSGeo.SDF,OSGeo.SHP

This has been mentioned before - but somehow I didn't know it. Here is an older reference:

http://zacster.blogspot.ch/2008/10/mapguide-performance-tips.html


In the next few days I will look into SQLite files as an alternative to SDF. A first test seems to be promising - SQLite file was much faster than SDF.

Map 2013, SP2