52°North Spatial Information Research GmbH

exploring horizons

Blog

Still going strong

by 1 Comment

Filed Under: ILWIS

The last weeks I mainly wrote about the next generation of Ilwis. But this is only part of my work, the smaller part even. The main work still goes to Ilwis 3. Undoubtedly there will come a point when the work on Ilwis 3 will only be maintenance but that is still some way off in the future. So what have I been doing in in Ilwis 3? Four main things

  1. Integrating parts of the Orfeo library into Ilwis
  2. Integrating WFS import into Ilwis
  3. Extending the ILWIS WPS
  4. Debugging

The main part of my work has been the integration of the Orfeo library into Ilwis. Orfeo is a big remote sensing library of CNES (http://www.orfeo-toolbox.org/otb/) that I wanted to be available in ILWIS for some time. Not that I need or want everything from that library but there are still a lot of routines in there that are useful to have available in Ilwis. The planning at the moment is that the following becomes available in the Ilwis 3.8.4

  • Vegegation indexes (~10 routines)
  • Atmospheric correction
  • Image fusion
  • Cloud detecttion
  • Road extraction
  • Image segmentation
  • Some unsupervised classification schemes
  • Unmixing

more >

MapCalc 2.0 and TabCalc 2.0

by Leave a Comment

Filed Under: ILWIS Tagged With: ,

Map/TabCalc was and will be a central part of ILWIS. In ILWIS 3 it is a very compact way to do calculations on whole data sets at once. It is easy to learn and forms the basis for many scripts that access ILWIS 3 functionality. However, it is far from perfect.

  • There is a confusing set of interfaces with different names and different subsets of the command set. You have the commandline, simplecalc, mapcalc, tabcalc, script; not easy to explain the differences between them. Is this really needed? I don’t think so.
  • There is a severe lack of conditionals and looping in the script. That’s ok if you have the IFF constructs, but they only work in a single call. There is no IF/THEN/ELSE. There is no WHILE. This severely limits the control you need for a script. Now you might make a case that you don’t need that since you can integrate ILWIS with the ILWIS-C commandline construct in almost any existing script environment. I do sympathize somewhat with this argument, but I don’t like a mandatory dependency on some external scripting environment (e.g. python). I do need to make a scripting environment for backward compatibility anyway, so improving it is not that big a deal.
  • The implementation of the current scripting is a mess. Its performance is ok. It does things reasonably efficiently, but it is one of the very few parts of the code in which I would rather not make any changes. The inner workings of it are rather obscure. As far as I know (the core of it was written before I started with ILWIS), there is no formal syntax with nice EBNF notation that runs through a parser/lexer generator combination to generate some abstract syntax tree or equivalent. It is a meticulously, hand crafted parser/lexer that does all the (very) complicated stuff that is normally obscured from the programmer and you must understand it if you want to make changes. I don’t expect that many readers have ever made custom languages (I have), but that is a horrible way to do it. As programmer, I only want to deal with the EBNF and (in this case) the abstract syntax tree. The unreadable code that connects these two is automatically generated with well know techniques and proven tools (that was also the case 18 years ago. I am a bit puzzled why those tools were not used).

more >

Into the Core

by Leave a Comment

Filed Under: ILWIS Tagged With: , , ,

The core of each GIS is of course built around the spatial data (duh). So how we access that data (on a programming level) should be of prime importance when designing Ilwis-Objects. Some people would argue that the organization of data is of prime importance but I don’t agree with that. Organization should be  a technical derivative of how I want to access my data. I don’t care if there is a database behind my interfaces or a file based system or a service or…. I just want to do things with my data. Organization is important, but only insofar it serves the interfaces.
So what  do I require of my interfaces?

  1. They must be simple! ( remember KISS). This is probably the hardest point. Paradoxical as it may seem but it is far easier to develop a complex set of interfaces than to develop a simple one. Complexity breeds complexity and thus grows at a very fast pace.
  2. Things that are the same should be the same. This is a cryptic point as it states something obvious. Still, I seldom see a very rigorous implementation of this in the software I am familiar with. E.g. is a (Ilwis3) maplist the same as a rastermap were only the nature of the pixel value is different ( a array instead  of a single value)? Arguably yes, so why are there different classes? It gets more interesting with vector maps. Is a point different from a polygon or are two spatial different points always different things? I will investigate this one further down below
  3. Natural integration within the C++ STL style of programming. The STL is a core library of C++ that (among other things) abstracts algorithms from data types. The reason why this is important is that it generalizes and simplifies the way how we can write algorithms tremendously. For example I rewrote the resample algorithm from over 150 lines of code to 8 lines of code.

more >

Plus ça change, plus c’est la même chose

by Leave a Comment

Filed Under: ILWIS Tagged With: , , , ,

Last week I talked a bit about what connectors are supposed to do. As mentioned there, their main task was to translate external representations to internal ones. This week I want to focus a bit on the internal representation. After all, that is what people will use when programming with ILWIS-Objects.

The design of the internal representation leans heavily in some ways on the existing implementation; in other ways, it deviates sharply from that. It was/is maybe somewhat surprising that an 18 year old design is still very relevant these days. At a high level, the data structure has not changed that much. We still talk about data layers and coordinate systems, the problem of the data’s semantics still has to be addressed and tables are still tables. What has changed is the nature of the data itself. Where 2D data was sufficient in the past, we now talk about 3D, 4D and even 5D data. We see the size and availability of data sets explode and we see the complexity of relations between data sharply increasing. So the claim is, while the nature of data certainly has changed, the overall structure has not changed as much.

more >