Last week I wrote about all the things we wanted to achieve for the next version of ILWIS. The question is of course, how realistic is this? I mean, the current ILWIS is collaberative effort over many years. There is a huge amount of knowledge, experience and work involved that is not easily reproducable. The original group that produced ILWIS doesn’t exist anymore (I was part of it) and the resources have been severely reduced. On the other hand, in the landscape of software development, there have been fundamental changes that are very beneficial to this undertaking. The rise of open source software is a very big boon and the frameworks for building UI’s have become much more convenient to use. And, to my (sort of at least) surprise, the core design of the ILWIS 2&3 data structures was, with some adaptations, still very usuable in the modern context; a tribute to the original designers (no, not me, before my time).

It is this organization of data and its changes that I want to talk about in this blog, not the nitty, gritty C++ that is the core of Ilwis, but more on the level that users will encounter. After all, this is the basis on which the rest of ILWIS is built.

So lets start with the maps (I use the word coverage for it these days, because the word map is more closely linked to the cartographic concept).

Raster coverage

This is mostely the same. It still has a georeference, coordinate system and a domain (more about that later). The big difference here is that maplists are gone and have merged with the raster coverage. A raster can have multiple bands by default and all algorithms work on multiple bands. You can of course retrieve single bands of it. I decided (though it would have been possible in the new data structure) not to go higher than 3 dimensions as the use cases for that are limited. In the end, you have to weigh the effort against the use and that fixed the case for 3 dimensions. The nice aspect of the “z” dimension is that it has its own domain (by default a ‘count’ domain) and so it easy to define spatial temporal multi band rasters, or multi spectral and stay in the semantics of those domains when defining operations on the raster.

Vector maps

Here there are significant changes.

• There are no point, segment or polygon maps anymore. The “feature coverage”, as the type is called, can contain all types simultaneously. These are maps with points, lines and polygons at the same time. This of course doesn’t prohibit coverages that are “mono” coverages. Many external formats provide data like that. But from a conceptual level Ilwis-objects can integrate them in one coverage.
• There is no more domain for feature coverages. In ILWIS 3, there was a very direct link between a domain, the map and the attribute table. As such, the link wasn’t bad and it was even logical, but the constraints it laid on the attribute table were less desired (e.g. the number of records in the table and the domain). Now there is a direct relation between the feature and the its attributes, just like e.g. ArcGIS does it. From a purely technical point of view, there isn’t even an attribute table anymore as every feature has its set’s attributes. The front end allows you to view these attributes as a table, but that is just a view.
• Similar to rasters, features are multidimensional, or maybe more correctly said, features can contain other “levels” of features. This sounds complex and it can be complex. Let me give an example. Suppose you have a world map and you represent a city on that. A point representation seems reasonable: On that scale, points are good enough and simple to draw. Now we zoom, let’s say, to the country level. Cities are now small polygons. It is still the same city, same attributes, same identifier, but its representation has changed. Now we zoom further to the municipal level. The city has now become a collection of points, polgyons and lines; still the same city though. Here is another example. Take a big iceberg that breaks off of one of the antarctic shelfs. Until it breaksup into smaller icebergs, we can track its movement as one feature. Its “levels” are simply snapshots in time, but at each snapshot it probably has a different shape and location (drifting, melting and its effects). In theory, I can represent the whole world over time in one feature. I guess my hard drive is too small though :). For those who think this can get way too complex (and they are correct), the base form looks very similar to a “normal” flat vector map and you don’t notice all the extra complexity you potentially can introduce.

Domains.

Yeah, they still exist. Though I wasn’t always happy with the rigidity they enforced, the concept behind it, enforcing semantics, is correct. So what has changed? I hope I have significantly recued the rigidity in the system. In the numerical domains, a large part of the rigidity was caused by something most users weren’t aware of. Numbers were internally stored in a special way (not going into details here). This caused a number of constraints on how data was stored and this reflected in the rigidity of the numerical domains. That is now gone and that made numerical domains much more “convertable”.

On the identifier/class side, we also had a problem. How can we compare class domains and why don’t we have a subset of domains. In Ilwis-objects this is resolved by the introduction of parent and child domains. A child domain is a subset of a parent domain which guarantees that they have the same semantics. You can now compare thematically similar maps with different domains (partially overlapping maybe) as long as the domains share the same parent domain. The principle also exists with numerical domain, but plays less of a role there.

Tables

They have hardly changed. Internally yes, but (for the moment) for the user not. I might introduce multi-dimensional tables, but only when I see a good use case. The only thing that doesn’t exist anymore is that tables are based on a domain. If you need this behaviour, simply indicate a column as the primary key and that has pretty much the same effect.

There have been many other changes of course, but this more or less sums it up for the main data types.