Last week I wrote about the high level design and structure of the next generation of Ilwis. This week I will focus on one of the most important parts of it, the connectors. In the picture of last week (part of it below), they made up the whole lower part of the second diagram

So they must be kind of important. And yes they are.  Connectors, as the name implies, connect ‘things’ to the Ilwis-Objects.  I write ‘things’ on purpose here. It is a bit vague, a bit undefined, while the other side is very specific –  Ilwis-Objects. And this is exactly what the purpose is. The physical nature of, for example, data-sources must have no impact on its logical representation. This is also true for location as its native internal data representation.

What does this mean? Well, if you work with data, should you care if data comes from a GeoTiff or from an Erdas Imagine file? Should you care if this data is on your local system or from some web service? In the end, you want to work with content, you want to work with the reflectances, the NDVI’s or whatever is logically present in that data. Its external (outside the framework) representation is basically irrelevant.

The same case can be made for operations.  It is not that operations are easily interchangeable. Image fusion from one library may work differently from Image fusion from another tool box. I am not going to solve the semantics problem, but  the access methods for operations can be harmonized. That a WPS method is remotely accessed by some complicated XML structure or URL, while a call to the Orfeo library involves a number of filters to be defined is a question of external representation (on a functional level) of those services/libraries, not of the logical representation of what you want to achieve. Same in both cases, you want some operation to run on your data. If we can abstract from this, we are also independent of the actual nature of the processing source.

Apart from the question of external/internal representation and location, there are also a number of other questions that need to be addressed, questions that are often solved at the ‘local’ level of the service/library, but where representation must be lifted beyond the local level to function properly in the next generation of Ilwis. Which questions? Well a number of them comes to mind: synchronous/ asynchronous data (e.g. sensors), progress indicators, error/warning messages, distributed processing.

All this is incorporated in the design of the connectors. Now if we do this correctly it may be clear that functionality wise, we don’t have to program much for Ilwis. We simply put a connector between us and the external data/processing and we are done. The trick is of course to define a fairly simple mechanism to translate external representations to internal ones and to have a sufficiently rich internal representation that can handle the semantics of most (all?) of the external representations. Don’t expect magic though, if the external library has its limitation (and most have) this will be mirrored in what is possible in the internal representation.

So what does this mean for the next generation of Ilwis?

• We don’t care what the data format is. A raster is raster because it is a raster and not because it is IDL format or HDF5 or whatever.
• We don’t care where the data is. A local polygon layer is not functionally different from a remotely accessed polygon layer.
• We don’t care what the nature of processing is. We do care about its semantics, but not how different processing sources interface with the external world.

This doesn’t mean that we, as a user, for example, don’t know where the data is. Of course we know, but this is ‘meta’ – knowledge. It doesn’t impact what we can do with the data.

Though this might seem a bit ambitious it really isn’t. It all relies on having the correct design. At the moment (2013-04-15) I have written  a set of connectors to the ‘old’ Ilwis3 format, a connector for gdal and a connector to a processing library for my test suite. Apart from the synchronous/asynchronous part, they behave as described above (the latter part will come, but the design is not finished yet).  It is a proof of concept and it will form the roots of the Ilwis-Objects framework.

Next week I will give a bit more  details about the Ilwis-Objects layer of the framework.