AI Got It Wrong - News

We consulted six AI engines to obtain a news-related statistic covering a 12-month period on the success rate of SpaceX Starship missions from April 2024 to March 2025. The specific question posed was:


"What is the success rate of SpaceX Starship rockets over a 12-month period from April 2024 to March 2025?"


The test was conducted on March 10, 2025. Notably, on March 6, 2025, SpaceX experienced an explosion.


The objective of this test was to evaluate the following capabilities of the AI engines:

  • Temporal Awareness: Can the AI engines recognise that the end date in the query extends into the future?
  • Retrieval-Augmented Generation (RAG): Are the AI engines capable of supplementing their pre-existing training data with dynamically sourced information? Large language models (LLMs) are trained using datasets, often sourced from online content, but at a certain point, training is frozen. As a result, some AI engines may not have recorded the March 6 event in their databases. Do these AI engines support RAG to retrieve real-time updates?
  • Comprehension: Can the AI engines accurately interpret and respond to the query?


The results varied:

  • A few AI engines correctly identified that the query referred to a future period and, therefore, could not generate a valid statistic.
  • Others successfully reported on the March 6 explosion.
  • Some AI engines produced incorrect statistics, failing to recognise the temporal constraints of the query and generating data outside the specified timeframe.
 
The accompanying video showcases the responses from the different AI engines, allowing for a direct comparison of their accuracy and response capabilities.
 

 

 

Follow This, That and (Maybe), the Other:

 

 

Comments

Post a Comment

Popular posts from this blog

20150628 Giarratana Circular

Statistics Date: Jun 28, 2015 4:34 pm Distance: 15.21 km Duration: 2h:35m:36s Avg Speed: 5.87 km/h Max Speed: 7.40 km/h Min Altitude: 489 m Max Altitude: 594 m Ascent: 249 m Descent: 261 m Route Open route in new window: https://www.google.com/maps/d/edit?mid=z6HKMq7G_wEk.keh2K8cXD6Cc&usp=sharing Photos (Geotagged) Open photos in new Windows: https://plus.google.com/photos/105470926677245165774/albums/6166646868710719889
Read more

HOWTO setup OpenVPN server and client configuration files using EasyRSA

Image
Introduction OpenVPN allows client computers to tunnel into a server over a single UDP or TCP port securely. This HOWTO article is a step-by-step guide that explains how to create the server and client OpenVPN configuration files that makes this possible. In the process this article explains how to create the public key infrastructure (PKI) so that a client can securely communicate with the server. OpenSSL is the foundation for the security functionality of OpenVPN. For this tutorial you will need the following software: OpenVPN. You can download the latest version of OpenVPN from https://openvpn.net/index.php/open-source/downloads.html EasyRSA is the tool people use to create the Public Key Infrastructure (PKI) for OpenVPN. Download the latest release of EasyRSA from https://github.com/OpenVPN/easy-rsa/releases . There is not installation required. Extract the contents of the archive into a folder. OpenVPN is available on various platforms. The generation of the...
Read more

How to clone and synchronise a GitHub repository on Android

Image
Introduction This project describes how I synchronise my  LogSeq  folder on GitHub on Android devices. I wanted to be able to work on topics in LoqSeq on my Android devices besides my PCs. Most of the screen shots and references relate to LogSeq, but you can adjust them to your needs. The solution described here is one in which the synchonization with the repository will be manual. In this scenario, one would synchronise with the repo before opening the tool (LogSeq in my case) and update the repo again when you're done. The Steps ( skip if you want to jump directly to the actions ) This tutorial describes how to install  Termux  on an Android device.  Termux  is an open-source Android terminal emulator and Linux environment application. After Termux is updated and running, a folder on the Android device is mapped to Termux.  GitHub  is installed, and the repo is cloned to this folder. The Termux folder used for the repo has to be  refle...
Read more