The second Beta of ILWIS 3.8 is done and on the download site!

So what can you expect of this Beta? Well, a number of things that were not available in the first beta are now available, such as color composite and some editors. Some new things, things that were not possible in 3.7 have been included. For example Hovmoller diagrams and base maps (see below).

Some things didn’t make it (yet), like the stereo pair and the sample set editor. The stereo pair is for a large part done and a first version will be done this week but I didn’t want to wait for it. The sample set editor is still in the start phase.

A few of the new things are:

Basemaps

Often it is convenient to have some data-sets always available; for example a boundary map of a certain region to make orientation easier. To facilitate this the “basemaps” have been introduced. Basically it is the same as “system” objects for ILWIS but this time for maps. They reside in the basemaps folder of the system folder of ILWIS. Any map in there will show up as base map when you want to add new layers. The add layers form has been adjusted to reflect this and in the Catalog there is a new menu entry under the “Show data” item (also in the context menu of the catalog). By default I have included three data sets to illustrate how this option works.

Color composites

Dynamic color composites were not possible in 3.8 beta 1. The editor for this has been fully rewritten and is now included in the 3.8 beta-2. It follows the same ideas as the other display tools. Double clicking on the nodes of the tree brings forth forms to change the properties of the tool (in this case stretch on the values and order of the bands). We didnt include HSI color composites yet as we were a bit unsure if this option was used often but we can include it.

Full screen mode

The content of the map window can now be shown in fullscreen mode. So no Layer tree view, no menu, only the map. There still a little bug(they are visible) with two scrollbars but that will be removed.

Hovmoller diagrams

A Hovmöller diagram is a commonly used way of plotting meteorological data to highlight the role of regular occuring events (“waves”). They are not limited to meteorological events though. In ILWIS the content of a maplist (over a longer period) is plotted on a tracking line on the map leading to diagrams like this:

This is the result of a maplist of 600 maps that cover the ndvi vallues of a part of Africa between 1982 and 1999.

Of course a number of bugs have been removed and the UI has been cleaned up at a number of places. I think this will be the last beta. The next version will be the release version.

With the 3.8 we tried to make the response of the the renderings to the users actions as direct as possible ( of course, where reasonable and technically feasible). With this I mean that if a user changes lets say the transparency of a layer, the transparency must change on the fly and not after pressing some apply button. The idea is to give immediate feedback and limmit the number of user interface actions a user needs to do when setting up a rendering. In this blog entry I want to show you a few of these.

An example : Interactive representation
In a previous blog entry I described how a number of the ideas for the ILWIS 3.8 mapwindow came from the ANIMVIS prototype. One of the important parts was there to be able to adapt the coloring of maps during an animation to be able to dynamically single out important areas on the maps with distinct value ranges. In fact a form of interactive slicing. When desigining and testing the required interface for ILWIS (ANIMVIS was a stand alone, non ILWIS prototype) it quickly grew in something more. Usefull not only for animations but also for single value maps and because I allowed gradual coloring of the slicing bands it was more than just a slicing tool. It had become an interactive representation.

So how does it work?

The default looks aren’t that impressive. It  just divides the map into two equal areas. One contains all the values above half of the value range, the other the rest. A basic slicing of the map. You may divide it into more areas by selecting different numbers in the combobox below. The fun begins when you click in the rectangle with the colors on the form on the dividing line between two areas. A thin red line forms and when you slowly drag that line (keep the left mouse button down) up or down you see the areas on the map change with it.  Below is an example of that could look like

I made three areas and divided into such a way that I get three areas that are divided into ranges I find usefull to be able to identify specific areas that match these criteria.  Infact if have a form of interactive slicing.

All this is great of course but it’s rather, ehm, green and well the moving of the boundaries is great for quickly locating key areas but it is not very precise. Fortunately there is a solution for this. When double clicking on an area (on the form, not the map), you get a form that gives you more control

with it you can, within one band change the low and high color, exactly define the boundaries and setting the transparency for that layer. Combining those properties I made the rendering below. 3 bands, one with a flat color, one with a gradual color and one with a flat color but 90% transparent.

with this method you can quickly visualy mark your areas which you find interesting  while leaving the rest of the map simple (flat colored) to avoid visual cluttering. It becomes realy interesting when this used in an animation but I leave that up to the reader to try out .

Example 2 : Track profile

The profile tracker. Profile trackers are of course not new but I added a few small things to it. The profile tracker is not by default enabled and you have to call it forth from the context menu on the display options node in the Layer treeview. After that you can click anywhere on the map and draw a line to any other location on the map. When you click again, the line is finished and the profile shown

The graph shows now the profile along the line. As you can see there are two line here. That is because I added a second data source for this profile. You can do this by double clicking on the node in the Layer treeview. Another small thing is that if you click on the graph. The exact value of that location will be shown in the list below it and a marker will appear in the map (rather tiny marker, maybe I will make it bigger). Last small thing is that if you press control when marking the end of the line it will continue the line until you end it. This way you can see the profile over a multi legged line.

One last remark. The track profile works just fine with class maps

A multi leg profile over a class map ( I increased the line thickness for more contrast).

You should see the track profile in an animation, there the line moves in sync with animation.

Other tools that work interactively are : stretching, transparency, colorcomposite and few others.

One of the goals of 3.8 was to improve the handling of spatial temporal data. Now the spatial part in ILWIS was of course ok, but the temporal part? In fact, besides some artificial  constructs, ILWIS had no knowledge of time. This makes the handling of data in the temporal domain much harder. So I designed a new Domain : Time.

Domain Time is meant to handle all things related to dates, hourse, seconds etc… It is a derivative of domain Value and at all places were domain Value can be used, domain Time can also be used. This is also true for MapCalc and TabCalc. So it quite possible to add two times together in MapCalc (and also do some operations that make less sense, dividing for example), or deduct times.

Internally Time is based on the astronomical definition of the Julian day (see also http://en.wikipedia.org/wiki/Julian_day). You won’t notice this on the “outside”, as a user, all dates will be represented in a standard format (see below). The Julian day is a day counting system with its zero point in January first, 4713 BC, Greenwich Noon. From that day (noon) all time are calculated. So today (2011-10-17) is  2455851 in Julian days (you can add fractionals to express hours, minutes, seconds). Though this representation is of course not very helpfull for us normal humans and thus is hidden in the user interface, it has one implication that is important ( in MapCalc and TabCalc), the unit of measurement is the day.

The second standard Domain Time is based upon is ISO-8601 ( see also http://en.wikipedia.org/wiki/Iso_time). This is the representation of time followed by the software and also the way ILWIS expects times to be entered. World wide and particular in software, the handling of date/time is very variable with respect to representation. Different calendars, different time standards, different languages makes it all quite messy. To simplify this for both the user and myself as implementor I use only one standard : ISO-8601. All times must be represented in that format and no other. Maybe a bit restrictive but I feel that extra flexibility makes the whole system less transparent. Also the notions of Periods and Durations is comming from that standard. If at any place I don’t support the standard, it’s a bug. So I don’t support times like Mon 17/7/2009 or 15-3-2004. They should be 2009-7-17 and 2005-3-15  or some ISO variants of this. ISO has some extra variations like 20090717 and such that are also acceptable. Read the linked wikepedia page for a more complete description.

So how do we make a DomainTime? The Create Domain form has been adapted

A new item is in the form with which to define the period for which a domain time is valid. In this case it is valid from 2001 the 15th of april 7 hour in the morning up until 31th of december in 2012. All times outside this period get an undefined value. There is also a default time domain (system) that runs from the beginning of the universe until its end (whenever that will be), should be sufficient.

In the program itself it looks like

Time also (as expected) resurfaces with animations. There is there a tab “real time progress” in which you can indicated which column (DomainTime of course) of the attribute table of a maplist is to be used in the animation as time progress. But wait, “attribute table of the maplist”?. MapLists don’t have attribute tables! Well actually, in 3.8 both MapLists and ObjectCollections may have attribute tables. The link between the table and the list is simply the index number of the map.

In the first tab you select the column to be used and define which period every second should be traversed. So in this case : 14 days, 7 hourse is “unit” per second. The animations add this period to an internal timer every second and switch to a new frame when the time defined for that frame (as per attribute column) is passed. The second tab show the progress through time ( and yes there is still a bug there ). Animations can also be synchr0nized using “real” time though some more testing would be neede there.

How do you create time columns? The obvious answer would be “by hand”, and this works of course. But when you have a maplist of 156 bands you won’t get very happy with that. Fortunately, often RS data comes with long, unwieldly names where the time information is somewhere in the 200 character long name. I have added an operation “Create Time Column” that parses the information out of that name. Due to the many different ways this information is recorded in the name the user has to indicate where the “parts” are to be found but that is rather trivial. The operation form looks like

If no time information is available, you are out luck. I can’t do magic.

Now the defintion of time is very new to ILWIS. Older stuff may not always support it or display it as a number (it is, after all, a domain value). I think I have nailed down most places (e.g. tables) but ILWIS is rather big, so it always possible that I missed a place.

I feel that the addition of this Domain is important. Time becomes increasingle important when studying dynamic datasets and with this addition a better modelling becomes possible.

This week I want to show some of small convenience additions in ILWIS 3.8. None of these changes have wide application like for example the animation or 3D features discussed in previous blogs but they add a level of convenience, a level of interactivity that was previously not there.
Particularly I want to show
- Filtering on class maps
- ObjectCollections as layer
- Map info pane

Filtering on Class maps.
When we display a class map usually we see all the classes. Sometimes though this is not what we need. Suppose I have an image and a class map of the same area and I want to understand the relationship between those two maps better. Each map covers the other so without running an application and filtering out some of the information it is difficult to do. Fortunately in ILWIS 3.8 we can do this quickly by simple “unchecking” class items in the legend tool in the Layer view.

For example, in the cut out below you see a landuse map covering an image of the same region but the polygonmap almost ompletely covers the raster map. In legend of  the representation you have the option to check out certain classes

If you for example check out : “Shrubs” and “Urban periphery” the result would look like this:

Nothing earth shattering, simply convenient.

ObjectCollections as Layer

Before 3.8 Object collections where not widely used in ILWIS. Intended as “virtual” folders, a convenient way of grouping your data without needing to create real folders all over your computer, many people found them a bit confusing. They looked too much like folders and could never be found as they were “virtual”. Anyway, in the 3.8 they see more uses. They are already the basis for animations for vector maps but they have other uses too. One of them is using them as a container that is intepreted as one layer in the map window.

Suppose I have collection of raster maps (value) that cover a large area. If I display them in the map window it could look like

12 maps with all different stretch values (they are independent). You might not see it but this actually a DTM of part of east africa. Due to the different stretch ranges this is almost impossible to recognize. You would have to adjust the stretch range of every individual map to get a decent rendering which is of course annoying. You could glue the maps together but what you don’t know that the actual area in this case is about 22000 x 16000 pixels. Glueing takes a lot of time then.
Fortunately I have a more convenient option in 3.8. If I organize these maps in an ObjectCollection I can show them as a single layer. The context menu of the collection will then look like

it has an open as layer options. Using that result would look like :

a correctly stretched, correctly positioned composite of all the maps in the container as a single continous layer. No mosaicing is done. It uses the ability of ILWIS 3.8 to reproject and reposition raster maps on the fly. As far as ILWIS is concerned this one single map though they are actualy still 12 seperate maps. All display tools that work for a single layer should als work for this composite. So things like the profile tool, dynamic representation or dynamic stretching work just fine.
The composite has one limitation, all maps in the collection must have the same domain. This is of course logical as the display tools see it as a single layer and a layer has one domain.
You will not use this option that often but it can be very convenient.

Map Info pane

Last I want to briefly show the Map Info pane. Actually this is the “old” pixel info. A very convenient tool that I felt was under utilized previously. It is located (default, you can move it around) at the bottom left and basically shows all the data of a location under the mouse cursor. This includes attribute tables, coordinate information, pixel location etc.. all.

It has a context menu with which you can customize it further. The continous check ensure that you get info dynamically following your mouse (when checked) or that only reports when you left click on the map.

Well, these are just three of the convenience tools. In ILWIS 3.8. There are many others but those have to wait until next blog.

One of the specific purposes of 3.8 was the improvement of the animations in ILWIS. These days there is so much spatial temportal data available and the amount of research directed at that is staggering. There are many ways to study spatial temporal data but one of the important ones is by animations.
ILWIS had a fairly primitive animations facility but that was not sufficient for the level of visual analysis I deemed necessary for ILWIS 3.8. Fortunately, here the ITC, there was some years ago, in the context of the reasearch work of C. Block ( http://www.itc.nl/personal/blok/), a prototype made by W. Nieuwenhuis of a tool called ANIMVIS. It displayed a lot of the properties I felt were necessary for an animation tool for visual analysis. I used these ideas as a basis for the work on animations.
So what were my goals? Well

• In all respects,were it made sense, it should work as a normal layer in the mapwindow. Meaning that an animations is just a layer and all the display properties that I can change for a “normal” layer should also work for the animation. E.g. Dynamic stretching works for a normal layer, so it should also work for the animation
• Animations of both raster en vector maps should be possible. The basic datastructure for raster is the maplist and for vector layers the object collection.
• The user must have full control of the dynamics of the animation ( start, stop, pause, frame by frame, speed etc…).
• Synchronization between different animations must be possible. Both on index basis as on “real time” basis (if available).