Fact Check and Investigate Weather News
In a previous blog post we had outlined the abilities to enhance weather related stories with what was possible with the first generation LLMs. Quite simply, LLMs were used to extract the intent (heatwave, flood, cold etc ) and entities (Rio de Janeiro, Nov 2023 etc) from the news article and a custom module within Weathersight retrieved data appropriate for adding context. The additions were certainly relevant but it was clear from the outset that it fell short of the guidance provided by a human who was well-versed with Weathersight’s capabilities.
In the last few months, we started re-imagining how the current generation of LLMs might influence the nature of assistants we add to Weathersight. The humans-with-expertise who added appropriate context to news stories often quickly figured out the subsets of data needed and used their familiarity with weather related datasets and tools to create the relevant information and visualizations. For example, on encountering an article about Western Europe's seemingly more intense and frequent heatwaves, a climate expert might say - “ah what we need is a trend line to show that frequency and intensity of 3 day heat-waves has gone up ; and the threshold for a heat-wave can be obtained by looking up 95th percentile of maximum temperatures during summer”. In short, the human-expert not only has domain expertise but the ability to understand the intent of the writer of the article - in this case to add relevant facts to support claims of more frequent and intense heatwaves.
We believed that with the rapid advances in LLM's and emergence of new protocols such as Claude’s ‘Tool Calling’ , ‘MCP’ and ‘Agent Loops’ - we could build a system that applied the expertise (of climate data scientists) to the news articles written by journalists to produce a fact-checked version of the original.
Weather Citation Agent
We have long observed that many weather news articles often lack the context necessary to communicate the rarity of a weather extreme or the shift in local weather patterns that led to it. The reasons for this are many - editors may omit the details because it is too verbose or confusing for the reader, or the pressure to publish quickly means that citations/links to claims made in the article are not researched in time. Sometimes the writers do not have access to subject matter experts or lack the expertise themselves to add the necessary context.
With the Citation Agent, a journalist or editor submits a weather related news article and gets back an augmented article with the following annotations:
Claims that are strongly or moderately supported. The supporting facts are added to the original article. The fact can appear as a chart/table or other visualization and is always accompanied by a citation to the dataset/API/webpage on Weathersight.
Claims that are not supported because contradicting facts were found.
Claims that are not supported because facts about them were not found.
The example below, is an article that appeared in a leading newspaper: The original text hasn’t been changed. The annotations (highlights in color) and the citations were added within minutes by the Citation Agent.
The visual (graph) has been generated by the Agent based on the APIs cited in citations 1,2.
In practice, we have observed that a few rounds of back-and-forth are needed to appropriately modify the original article. Indeed, this is the current area of development where we are experimenting with a few workflows and interfaces.
Investigation Agent
It is natural for a lot of investigations to start with a single line of enquiry and weather related news articles are no different. In fact a lot of content creators are already using general purpose AI assistants such as Claude / ChatGPT to acquire the information necessary for explaining and grounding a news story in fact.
AI assistants are only getting better at serving research needs. e.g. by providing explanations of weather phenomena (with references) for questions like ‘what is a heat-dome?’. While they do a reasonable job at retrieving and synthesizing ‘the most famous heat-domes in the past’ (this is expected as the articles on Wiki, research papers, old news articles are all part of their training data) , they still fall-short in retrieving and making sense of more specialized data (latest weather extremes, more esoteric computations such as return-periods, or plotting a trend of increasing heat-waves for a particular location).
However, with MCP (Model Context Protocol), it is possible for users of Claude to add a remote MCP connector to bring specialized datasets into their Claude sessions - in our case, any user of Claude (on mobile/desktop/browser) can get Claude to talk to Weathersight, all one needs is to sign up on Weathersight and follow the instructions to get a token. What does this interaction look like?
Claude automatically decides when it is appropriate to call Weathersight
In this example, one decides to learn about the impact of the early heatwave in Europe. Notice how Claude uses the registered connector (Weathersight) and calls all the relevant APIs dynamically to produce the desired result (the final table not shown for brevity).
You can explicitly ask Claude to use Weathersight
In this example, Claude dynamically combines information it knows from outside sources (e.g. locations of Tour de’ France in 2026) with WBGT (a human heat-stress indicator) available on Weathersight.
The output (condensed):
What’s Next
We are aiming to improve the performance (both recall and precision) of our Agents. For that, we need high quality data with high coverage. We also want Claude and other AI Agents to be able to retrieve the data quickly and consistently. These two initiatives are what's on our plate for the next few months.
Improve Data Coverage and Quality
We will continue to enhance the quality and coverage of Weathersight. Our commitment to observations is stronger than ever. While it is true that we have ERA5 data trends and analyses from 1951- , most of our efforts have been in extending our historical datasets - ~250 US locations have over a century of data (ThreadEx) and we have extended our coverage to nearly 4000 locations (from 3000) covering many locations that are also places of interest.
Rich API Support
We have increased our APIs from about 10 to nearly 20 - several of them have implemented advanced statistical functions that can be used to compute historical heat-waves or understand return-periods.
We are planning to add industry-specific APIs in the next few months - starting with a travel-centric API.
And yes, all those capabilities will be available immediately in the Investigation and Citation Agents. Stay tuned.