Exploring how Large Language Models can Improve Search in Spatial Data Infrastructures

Every day, a huge amount of spatial data is produced by public agencies, research institutes, companies, and other data providers. Spatial Data Infrastructures (SDIs) have been set up at local, national and international levels to bring these data and its metadata together and make it easier to find. Metadata standards have matured and been adopted along the way. And yet, users still struggle to find the right datasets.

Can you remember the last time you were looking for a dataset? Perhaps you had to visit several geoportals or open data portals, each with its own interface and layout. You typed a few keywords and hoped they matched something in the metadata. Woe unto you if you used a synonym! Searching for “global warming datasets” will not necessarily find a record described as “greenhouse effect data”. Your chances are even worse if you searched in English while the metadata was written in German, French or any other different language. And if you did get a keyword right, you probably faced a long, poorly ranked list of results and had to open each one and check whether it’s relevant.

If you can relate to these frustrations, you are not alone. Many users spend a lot of time searching for data before they can use it. This brings us to this week’s challenge:

How can Large Language Models (LLMs) improve data discovery in SDIs?

LLMs have become popular thanks to their advanced language processing abilities. You have probably used ChatGPT, Gemini, or Mistral to explain a concept, find some information, translate text, generate code or even build an app. There is no doubt that their capabilities are impressive. So why not just ask an LLM to find a particular dataset?

Unfortunately, that doesn’t work reliably. An LLM stores what it learned during training in its internal parameters. Its internal knowledge is fixed until the LLM is re-trained or updated, and until that happens, the model knows nothing about datasets published after its training period. Modern LLMs have internet search tools and can find popular, widely used datasets. However, many datasets in SDIs sit in catalogues that have not been indexed by search engines, so they remain invisible to the model.

Now ask an LLM for datasets  covering a very specific area, for instance, “Forest data for Hamburg”. Metadata usually describes a dataset’s spatial extent as bounding box coordinates, not as place names. LLMs are trained primarily on text and don’t always handle geographic reasoning well. Unless the place name is mentioned in the dataset’s title or description, the LLM struggles to translate bounding box coordinates to place names you used in your search query.

In short, using an LLM for data discovery has two limitations: outdated or incomplete knowledge and poor geographic reasoning.

Bringing the metadata to the model

An update to the model would be required with every newly published or updated dataset. Hence, re-training or fine-tuning the model are too time-consuming and resource-intensive approaches. Instead, when a user makes a request, we give the LLM immediate access to the latest metadata in an SDI. The model does not access all information at once to avoid overloading the context window. The model only accesses a subset of the information that is relevant to the user query. This approach is called Retrieval-Augmented Generation (RAG).

Figure 1 shows the main components of the system. The first building block is semantic search. Datasets are retrieved based on a similarity analysis of the user query and the metadata rather than exact matching of keywords. To make this work, both the user’s query and the metadata records are transformed into embeddings: vectors of numbers that encode what a text means. Two words with a similar meaning end up close together  even when they share no words at all. A request for global warming data can therefore find a record about the greenhouse effect. You can describe the dataset you are looking for in your own words, even in a different language from the metadata, and the system is more likely to find it.

But wait! What about the second problem, poor geographic reasoning? Embeddings are excellent at matching meaning, but they are not built for spatial reasoning. To them, the word “Münster” and a bounding box around 7.6° E, 52.0° N have nothing in common. This is where geographic matching comes in. Geocoding translates the place name mentioned in a query into coordinates, and spatial indexing compares those coordinates with the bounding boxes of the metadata. Only datasets whose extent fits the area of interest make it into the results.

Finally, the LLM receives the matching metadata records and uses them to answer the user’s query in plain language. It no longer has to rely only on what it learned during training.  The LLM can also re-rank the identified datasets to align them more closely to the user’s needs.

Here is how this could look in practice: you ask, in English, for “Renewable energy potential in Germany”. Semantic search finds records related to renewable energy, including those whose metadata talks about “Solarpotential” and “Geothermiepotential”. Geographic matching then keeps only the datasets whose bounding box overlaps with Germany. Instead of a long, unsorted list, you get a short list of datasets that fit both your topic and your area (see Figure 2). The response includes links to the actual datasets that you could then use or explore further.

Of course, this is no magic wand. The system can only find what the metadata describes. Good metadata remains the foundation of any SDI or data discovery workflow. Poor or vague metadata will make both the semantic search and the geographic matching less effective, and datasets can remain undiscovered.

Less searching, more analyzing

So, how can you bring this kind of search to your users? 52°North can develop a customized, AI-enabled data discovery solution for your SDI or data repository. The users of your data should spend less time hunting for data and more time where it matters: analyzing the data they find and deriving insights from it.

Do you have questions or are you curious how this would work with your own catalogue? Don’t hesitate to get in touch with us!

See you next week.