Sensor Web, Geoprocessing, Security & GeoRM, Geostatistics, Semantics, 3D, Ilwis, Earth Observation

Stacking rasters

November 29th, 2011 @ 10:15 by Martin Schouwenburg
Posted in ILWIS

It is always difficult when you have sets of different overlapping rasters to visualize properly. Rasters are usualy a full coverage and so two rasters(or more) on top of each other block the view on one of them. As historically ILWIS is oriented towards rasters  I felt I had to offer more flexibility when displaying rasters. So what options do we have? Well basically three

  1. Transparency. Make one layer (partially) transparent so you can view the “lower” layer
  2. 3D Orientation. seperate the two layers in 3D space and view them there
  3. Time shift. Quickly blink between layers to inspect different layers
So lets explore these options
Transparency and visibility
ILWIS 3.8 has fairly extensive transparency facilities. There is of course the “normal” full layer transparency. This applies transparency to each and every pixel in the map. It is fine but it has its limitations. The view you create has basically merged the pixels of two layers and it is not always that clear which features come from the upper or lower map.
I felt we needed a way to emphasize “interesting” pixels (whatever that maybe) in one map more and to filter out the rest.  Than you would see only partially one map, the interesting bits, and keep the rest of the underlying map. To do this ILWIS has two methods.
  • Class maps may make some of their class fully transparent. In the legend one can deselect classes which than become fully transparent. You get than a picture like this
Two classes have been deselected and the underlying map becomes visible.
  • A value range can become transparent. In the beta you already have the interactive representation (discussed in an earlier blog), in the internal version I have(2011-11-29) I have added two, related, extra options. The first one is directly through the tranparency form of a layer.

  • In this you see a range of values(0-150) being indicated as being 100% transparent and the underlying polygon map becomes visible. Related to this is the “command-line” method. In the internal version each mapwindow has a commandline that is similar to the command line of the mainwindow, Similar, but not 100% equal. Calculated maps from that command line are automatically added to the mapwindow. It has one extra option. The ability to add a displayoptions parameter that describes how the added map should be rendered. One of the parameters in there is similar to the transparency in the form. Below is such a result( with the calc expression)
3D orientation
Another way to reach the same goal is using 3D. It is not as flexible as the previous method but it might be usefull some cases. Below is a combination of the two methods with a 3D grid to get a better orientation.
 3D is more usefull in this respect when you have 3 or more layers (geologist studying layers in the earth use this view for example).
Blink
This is a special way to detect differences between two raster maps. It of course only relevant for two (or more) maps that are mostely the same but might differ on a number of places. One can make a small maplist with the maps, animate it and detect the differences.

Hidden in the crowd

November 21st, 2011 @ 09:36 by Martin Schouwenburg
Posted in ILWIS

ILWIS has I believe 200+ distinct operations, quite a number of them are actually containers for a whole bunch of tightly related similar operations. For example MapFilter which is actually a collection several dozen seperate filter operations. So how many operations does ILWIS “realy” contain? I don’t know, it depends, as I have shown above, on how you count; quite a lot I suppose.

Many operations have existed since a long time and are well  known to the ILWIS users. Still, new applications are being created and because for the moment the help files aren’t being updated ( can’t do everything by myself) they tend to hide in the shadows apart for the few users that have requested them. I was remembered of that when last week a colleague of my explained a complicated procedure to extract contour lines from a value map. So I thought it might be usefull to introduce  two of them.

Iso lines

Hidden in a corner of the SMCE was an operation with the rather opaque name SegmentMapFromRasValueBnd. Hardly anybody knew of it and it was not reachable from outside the SMCE. Basically it created an iso line map for certain value maps. The algorithm was based on the raster to polygon code but was somewhat limited and inflexible. So I lifted to code from the SMCE, moved it to the Segment application, adapted it and retained its horrible name (will rename it one of these days to something more descriptive lile SegmentMapIsolines).

The application calculates a segment map from a raster map with iso lines at defined intervals. There are two variants. One that calculates iso lines in a linear way. From a begin point to and end point with a set interval. The other lets the user define its intervals in a list like “500,1000,2000,3500” for example. So the intervals are totally under the users control. The application form looks like

The only option I think I want to add to the application is to define a minimum length of an Iso line ( in percentage??) to remove clutter and noise.

It’s a nice little application but a bit hidden.

MapChangeDetection

Change detection is whole field of study unto it self and I don’t claim that this application “solves” it. It simply calculates differences between maplists( domain value) and some other data source. It will always generate another maplist which contains the difference maps. It has three cases

  • Difference between one maplist and another maplist (equal lenght ofc). For example the ndvi values of one month in one year and the same month the next year.
  • Difference between one maplist and a base line (reference) map. For example temperature variations over a year with respect to the average yearly temperatures for a region.
  • Changes within the maplist to observe the actual change over a period.
Apart from that there are two additional parameters that apply to all three cases.
  • Threshold. Delta’s below this level are considered no change. This removes the clutter of a lot of micro changes.
  • Handling of undefs. Basically any difference between an undef and some value is of course undefined. But this is sometimes unwanted. The user has the option to consider undefined differences as being difference with a delta of zero.
The application form looks like
And yes, the label of the form is wrong :).
These are just two of the newer applications that are hidden in the crowd.

Pointology

November 14th, 2011 @ 10:49 by Martin Schouwenburg
Posted in ILWIS

One of the the things that has gotten a major overhaul in 3.8 is the way of how point maps are drawn. Before 3.8 it was a bit chaotic. You had multiple options there to drawn point maps with very different sets of attributes. For example, the “graph” drawing and the “true-type” drawing. For 3.8 I simplified the whole system and basically there is now only one way to draw points. As a consequence this means that there is only one set of attributes to change and that will work for all points of all types.

The basis of it all is the “symbol”. The symbol is the representation of a single point that is described in a “svg”-like text file located under resources\symbols in the installation folder. I say “svg’-like because the syntax of the description of the symbols is a derivative of a small subset of svg ( see http://en.wikipedia.org/wiki/Scalable_Vector_Graphics).  Svg is big and I selected what i felt I needed from it to define my own symbols and skipped the rest. The I added a few things that were needed to respond to attributes that the user could set. The result is a fairly small set of commands that can be used to define a symbol. Each symbol has its own file ( with an .ivg extension) and ILWIS reads them when the first pointmap needs to be drawn.

Before showing a bit what is possible with it, let me first a bit explain about the geometry of points in the map window. Every point symbol has default the same size. 1/200 th of the size of the zoomed in map window. This means that at every scale the point has the same size. The default size is multiplied by the “scale” of the point ( default 1.0) to enlarge/shrink it if needed. So there are no “point” sizes for symbols as there were previously. Each point symbol is centered around, eehm, its center 🙂 and scales it within the default size. ILWIS shifts each defintion of a symbol to the center of the bounding coordinate box that defines the lines, arcs and other drawing commands of the symbol. So it doesnt matter which units or relative position you use when defining the symbol, ILWIS makes sure it fits. If no sizes are defined (some simple drawing commands can do this, see below) ILWIS will default(size) to the default symbol size.

Though the default size is 1/200, ILWIS ,may adapt this default size if there are a lot(100+) of points to something smaller. Else the points would overcrowd the mapwindow. You can always overrule this by changing the scale as this default is only calculated when opening the map.

So back to the Ivg defintions. Lets look at a simple example. The definition of a simple right standing cross

<ivg scale="1.0" version="1.1">

<line x1="0" y1="-50" x2="0" y2="50"
style="stroke-width:1;stroke:black"/>

<line x1="-50" y1="0" x2="50" y2="0"
style="stroke-width:1;stroke:black"/>

</ivg>

This is straighforward SVG.  The cross consists of two crossing lines defined by x1,y1 and x2,y2 coordinates and a style definition. The style follows is a subset of the normal style definitions in SVG. I don’t support gradients (at least not yet) and some of the other more complex color definitions, just simple flat colors. A special color is ‘user-defined’, this is a color set from the Representation of ILWIS. I support the line, polygon, circle, rectangle and path. In the beta-2, the path didnt work correctly yet but that is solved now.

For more examples how to make your own symbol just look into the other ivg files in the symbols folder.

Apart from that ILWIS defines two special operations on the points : stretching and rotating on attribute values. Stretching stretches the scaling of a point based on its postion relative in the value range. Rotation does the same but then with the orientation of the symbol.

For the moment the the whole “mini-graph’ part is gone. I might reintroduce the pie-chart idea but for the others I don’t see much use to warant the time investement.

Busy

November 7th, 2011 @ 09:41 by Martin Schouwenburg
Posted in ILWIS

Last week I finally got the go-ahead for something I have been planning to do for some time. It’s not in the 3.8, but a first alpha of it will probably (I hope 🙂 ) not be far behind it. What am I talking about? Well, about Python (the programming language) and ILWIS. The intention is to make a module that extends Python with Geo-functionality based on the ILWIS libaries. First I want to expose the data interface and calculator of ILWIS in some way and maybe after that the rendering of ILWIS maps (not sure about that yet).

This extension I think is quite important as it makes it possible to connect ILWIS functionality to a lot of other tools that have also Python interfaces ( e.g. R). For many people Python is easier to program than C++ (the language ILWIS is written in) so I might get some more people contributing to ILWIS. It will probably also enable an easy way to integrate Python scripts in ILWIS. Though it opens up a lot of potential if I succeed, using it has some disadvantages. Python is considerably slower than C++, so writing heavy duty, high performace Python scripts is a pipe dream. But there are many other things where it can be useful so I am looking forward to it.

Yes we can

Last week a colleague of mine remarked that he found it strange that you couldn’t do operations in the Mapwindow like, for example, in ArcGIS (though that implementation is, like much in ArcGIS, a bit clunky). I never really thought about it, but I saw some merits in his observation. The idea is to be able to do an operation on a layer and the get the result directly in the mapwindow. For example, a buffer operation on a pointmap and then getting the buffered pointmap directly in the mapwindow.

After thinking a (very) short while about it I realized that I had all the routines needed for it already available and I only needed to make a new ‘Tool’ for the mapwindow that connected these routines. Within 2 hours I had the first prototype and after a little bit more debugging I had a working first version.

It looks like this

A new node has appeared in the Layer tree called “operations’. When you open the node you get the normal application dialog and when pressing OK, ILWIS starts calculating and the result is added to the layers.

For example, when running the Isolines applications I get the following

a new layer is automatically (iso) added.

Apart from the functionality, for me it was quite a nice exercise to see if the underlying design in ILWIS was flexible enough to adapt quickly to new whishes.

Anyway, back to debugging.