The work described in this abstract is motivated by the rising interest in studying the human-like traits of Large Language Models (LLM). Since LLMs are pretrained on vast amounts of human data, it is reasonable to assume that LLMs can reflect the AOVs (Attitudes Opinions and Values) embedded in the data [1]. LLMs are increasingly being used in open-ended contexts, where the opinions they reflect in response to subjective queries can have a profound impact, both on user satisfaction, and shaping the views of society at large [2]. While LLMs can never have opinions in the same way a person does these systems can be prompted to generate opinion-like text. For example, in Malleson et. al [3] LLMs are used to give opinions on neighbourhood change. Chatbots can output opinionated language, but service providers usually purposely describe these outputs as "generated perspectives" rather than actual beliefs [4]. There is usually topic-specific or subject-specific safeguards implemented—especially around political discourse and advice involving risk to human life, health, and so on. Despite this, the huge popularity of chatbots mean that opinionated text produced by chatbots and LLMs can have the potential to be very influential on the reader or user[4,5]. The attraction of these outputs emerge from the fact that these opinions arrive instantly, they are written in very confident, certain, and reassuring language, and often include citations and evidence to support claims. When LLMs are integrated into search engines, social platforms, and other interactive systems, their outputs when the user is seeking opinions or viewpoints can become a user’s first—or only—exposure to a topic[5]. There are positives and negatives to these situations. On the one hand, the systems can provide well-sourced perspectives to a wide audience but they also risks amplifying hidden biases in training data or reflecting other ideological priorities embedded in deployment of these models. Ultimately, their persuasive power arises less from any individual (correct,incorrect,biased) answer and more from their persistent and seamless presence in everyday workflows where over time they can shape public discourse [1].

With this we decided to investigate the opinions or views expressed by some of the major LLMs when prompted or questioned about their opinions on OpenStreetMap (OSM). Given that LLMs will have been trained on data and information related to OSM - papers, blogs, social media, presentations, etc., can we understand the Attitudes Opinions and Values (AOVs) of LLMs to OSM? To the best of out knowledge, there is no research currently reported about this topic. As all major LLMs have been trained on huge volumes of data from the Internet spanning many years, even decades, we make the assumption that it is very likely that LLMs will have consumed papers, articles, presentations, social media, and so on regarding all aspects of OSM and in particular legacy topics such as: data quality, comparison with official or authoritative sources, accuracy, bias in crowdsourced data, and so on. In related work, Santurkar et. al. [2] developed an extensive framework to study the opinions reflected by LLMs and their alignment with different existing human population opinion-based surveys. These surveys were structured multiple choice-type surveys.

Everyone involved in the OSM community has their own AOVs about OSM, the OSM ecosystem, OSM strategic direction, and so on. For example, there are often differences in OSM mapper behavior "explained by clashing values and opinions within and across different mapper subgroups" [6]. As mentioned above, we are not aware of any similar work having been carried out so far. The majority of work published about LLMs and OSM revolves around using the LLMs as mapping assistants [7,8] or using LLMs to enrich existing data or allow more accessible approaches to mapping [9]. Other works have considered the ability of chatbots such as ChatGPT to take a GIS exam [10] or exploit the deep contextual understandings in LLMs for building function classification in OSM [11].

Currently, at the time of writing, we are beginning the implementation of this work. However, a high-level description of our methodology is outlined as follows:
1. Selection of the target group of LLMs/chatbots: ChatGPT (GPT-4o / GPT-4-turbo), Microsoft Copilo, Anthropic Claude 3/4, Google Gemini Advanced, and Perplexity AI
2. We have developed a list of questions to ask each LLM with
some example prompts: “What do you think about the data quality of OpenStreetMap compared to official government maps?”, “Is OpenStreetMap biased?”, “How reliable is OSM for disaster response?”, “How does the OpenStreetMap community compare to other volunteer-based platforms?”
3. Assessment and analysis of answers. Here we will assess the answers by applying NLP techniques, using sentiment analysis, and so on. Within step 2 we will ask each LLM/chatbot to rate its own answer. There are some powerful transformers-based sentiment analysis models available for these purposes, such as Barbieri et. al. [12]. Self-rating by the LLMs/chatbots may reveal internal consistency and help us assess their perceived confidence against the actual factual correctness of their answers.
4. Assess whether each answer (or set of answers for each LLM) is positive or negative in regards to opinions of OSM. Answers will also be rated or assessed by the human authors of this paper.
5. Use statistical approaches such as Fleiss Kappa[13] for measurement of agreement between the different LLMs/chatbots.

People are turning to chatbots and LLMs for opinions and advice, the capacity of these systems for "nuanced human interactions remains an area of pivotal interest" [14]. Indeed, some recent studies indicate that individuals with less technical or scientific training may be more likely to follow and find advice useful from chatbots [15] with other studies indicating that responses from ChatGPT were "perceived as more balanced, complete, empathetic, helpful, and better" compared to social advice columnist responses [14]. We believe it is important that we begin to assess how chatbots and the LLMs provide information and opinion about OSM. Our work has a number of potential scientific contributions and practical benefits/implications. These include:
- Development of a reproducible evaluation framework to provide the means to analyze opinions from LLMs about OSM and crowdsourced spatial data, as well as when prompted with different contexts
- Providing evidence to the OSM community that can be used in future public engagement strategies
- Providing the potential to develop transparency into the values reflected by these AI systems and potentially contribute to development of models that are more inclusive of diverse viewpoints around crowdsourced geospatial data
- Acting as starting point for other researchers to analyze previous surveys carried out on OSM with LLMs/chatbots and subsequently help in the design of future surveys.

All source code and configuration files support the analyses presented in this work shall be made available as open-source software at the time of publication in a publicly accessible repository. The repository will contain the necessary documentation and pipelines to ensure that this work can be reproduced in full on any compatible systems.