Turning collections data checks into an AI conversation

As part of the Digital Collections team, Collections Data Manager Gareth Watkins supports kaimahi across the museum to maintain accurate information in EMu, Te Papa’s Collection Management System, ensuring it supports the documentation and management of more than a million collection items, taonga and natural history specimens. Here, Gareth shares how he created an AI-friendly report that lets kaimahi ask questions about the collections data and find the information most relevant to their work.

In an earlier blog, I wrote about the quarterly collections data checks introduced in April 2024 to help keep Te Papa’s collections information in good health. There are now more than ninety tests covering areas such as registration, locations, object classification, taxonomy, imaging and publication.

Each quarter, these data checks produce a large amount of useful information. The challenge is helping kaimahi understand what the numbers mean and find the results most relevant to their work. Collection managers, curators and senior leaders will each have different questions.

To make the results more useful, I recently designed an AI-friendly plain-text report that can be uploaded to Microsoft Copilot, operating within Te Papa’s managed Microsoft environment. Kaimahi can then explore the quarterly results by asking questions relevant to their work, rather than viewing the data through a single, predetermined lens.

Building context into the file

To give the AI model context, I designed the report to include the latest results, earlier quarters for identifying trends, descriptions of each test and broader information such as Te Papa’s Statement of Intent.

This contextual information is crucial for helping the model understand the numbers. Some tests identify opportunities rather than errors, while others naturally return larger numbers because of a collection’s size or nature.

The report explicitly instructs the model to use only the supplied information, explain unfamiliar tests, avoid inventing reasons for results and acknowledge when a question cannot be answered.

The report also maps the results to Ngā Tikanga Whakahaere Kohinga Taonga a Te Papa / The Collection Care & Practice Framework, which describes standards across fifteen areas, including registration, storage, conservation and access.

This shifts the conversation from “How many records matched this test?” to “What might this tell us about registration, risk, digital access, audits or collection care?”

Designed for both people and machines

The report is generated using Python and saved as a Markdown-formatted text file. Markdown structures text using basic headings, lists and tables, making it easy for an AI model to interpret while remaining readable to people.

Keeping the report readable to people is especially important because human review of AI-generated output is a core requirement when using AI tools at Te Papa. The image below shows an example Markdown table:

The report contains only aggregated counts of records and issues. It does not include details about individual collection items or taonga, current locations, valuations or other sensitive information.

Aggregation also keeps the file small enough to fit easily within the model’s context window, which is the amount of information the AI can consider during a conversation.

What can kaimahi ask?

The real advantage with this way of reporting is that the AI model can tailor its responses to the questions and needs of the person using it.

Starter prompts might include:

For a curator: “I am a curator responsible for the Art collection. Explain the most relevant results for my collection, including meaningful changes from earlier quarters.”

For a leadership briefing: “Prepare a concise briefing for senior leaders. Focus on progress, emerging risks, digital access, and areas where targeted effort could make a difference.”

For a general overview: “Summarise the most important patterns in this quarter’s results and highlight notable changes over time.”

Kaimahi can then ask follow-up questions, reshape the response for different audiences or request a different form of output, such as an infographic.

Infographics

Visual summaries can be an excellent way of communicating results, but they require particular care. For example, I asked Copilot to create an infographic showing the number of items physically on display, split by collection.

The figures for Birds, Art, and Plants were accurate, but the graphic also included Performing Arts, Military History, and Textiles. These names did not appear in the source report and are not valid collection names in our collection management system. Relevant items are instead spread across collections such as History, Taonga Māori, Pacific Cultures, and Photography.

So where did those collection names and numbers come from?

When I asked Copilot to explain the result, it said that image generation was a secondary step and that it had filled perceived gaps with plausible-looking content.

I discovered that a more successful way of creating infographics was to split the process into two steps. First, get AI to identify the key points and numbers for the infographic, then create a new prompt containing the exact text and numbers that should be included in the generated image.

I think a key lesson here is to routinely challenge AI-generated results and ask the model to identify its sources of information or show the calculations it used.

Challenge, cross-check and verify

Reflecting on cross-checking and human verification, I have found that another approved AI model can provide a useful first-pass review of AI-generated content.

In the infographic example, a second model (via ChatGPT) quickly identified the collection names and figures that were not supported by the source report. However, agreement between two AI models is also not proof that something is correct, as both could overlook or repeat the same error. Ultimately, human verification remains essential, particularly when results may influence collection care and access activities.

With the AI-friendly report I have designed, AI is not making decisions about the collections, and it does not replace the knowledge of collection specialists. Its role is to help kaimahi navigate, compare and communicate information that is already there. The value of AI here is not that it knows more than us, but that it gives us practical new ways to engage with the information we already hold.