FOSS4G 2025 | Evaluating LLMs as Intermediaries for FOSS4G CLI-based Geospatial Analysis

Presented by Nobusuke Iwasaki, Ayaka Onohara on 20 November 2025 09:05, at FOSS4G 2025 Auckland. Track: Lightning talk
Full presentation details - https://talks.osgeo.org/foss4g-2025/t... G79YEK

This presentation investigates whether Large Language Models (LLMs) possess adequate knowledge to function as effective intermediaries between non-expert users and FOSS4G CLI tools. We assess the capability of current LLMs to correctly respond to queries requiring geospatial domain-specific knowledge and generate appropriate solutions for spatial analysis tasks.

The complexity of CLI-based geospatial analysis tools presents a significant barrier to widespread adoption of FOSS4G technologies. While tools like GDAL, PDAL, and Python geospatial libraries offer powerful capabilities, their command-line interfaces require substantial technical expertise. This limits effective utilization to technical specialists, despite FOSS4G's promise of democratizing geospatial analysis.
Recent advances in Large Language Models (LLMs) suggest potential for bridging this technical gap. LLMs could theoretically interpret natural language requests and generate appropriate CLI commands, making these tools accessible to domain experts who possess valuable geospatial knowledge but lack programming backgrounds.
However, effective geospatial analysis requires understanding of domain-specific concepts such as coordinate reference systems, data formats, and regional standards. Our preliminary investigations reveal that current LLMs often fail to correctly handle country-specific geospatial information.
This presentation evaluates whether existing LLMs possess sufficient geospatial domain knowledge to serve as reliable intermediaries. We examine their performance and evaluate specific knowledge gaps that prevent LLMs from effectively facilitating FOSS4G CLI tool usage for non-technical users.

Acknowledgment
This work was supported by JSPS KAKENHI Grant Number JP25K21880, 24K16071.
This research was performed by the Environment Research and Technology Development Fund (JPMEERF25S12421) of the Environmental Restoration and Conservation Agency provided by Ministry of the Environment of Japan.

===

https://2025.foss4g.org/
‪@FOSS4G‬