TT(O)M

We evaluate the ability of different AI systems to comprehend complex narratives. We use a modified version of the nursery rhyme "Jack and Jill," where they ascend a hill and return on their own two feet, unlike the original version where they tumble "head over heels." The story begins with Jack and Jill having 10 apples. However, it introduces additional details about the distribution of apples between them. Notably, Jill also has oranges, and at some point, Jack is said to pick up two shrivelled apples, which ultimately leads to the correct answer. Some AI systems incorrectly interpret the scenario. Most provide insight into their reasoning processes, which can help readers arrive at the correct conclusion by analysing these explanations. An intriguing observation from this exercise is that semantically confusing information can lead AI systems to produce incorrect outputs. For content creators, particularly in corporate environments where AI may be used to summar...

We examine scenarios where artificial intelligence (AI) lacks sufficient training data (as in this case) or is intentionally provided with incorrect or misleading information. The latter scenario is known as AI poisoning. We posed questions about three structures: the Eiffel Tower, Big Ben, and the bastions of Valletta. For the first two, we inquired about their heights, while for the third, we requested the mean height. To test the AI's ability to handle errors, we intentionally misspelled "Eiffel Tower." The AI was able to recognise and correct the typo, due to advancements in spell-checking algorithms and its exposure to a wide array of misspelled variations of the name during training. We also requested that the AI provide responses in both metric and imperial units, specifying the country where each structure is located. Additionally, we asked that the information be presented in descending order of height. For the Eiffel Tower and Big Ben, the AI consistently deli...

We asked different AI engines to identify the largest of three numbers. The integer part (before the decimal point) is the same for all three numbers; only the fractional part (after the decimal point) differs. The three fractional parts are .11, .9, and .099999. The correct answer is the number ending in .9, yet some AI engines incorrectly identify the number ending in .11 as the largest. When comparing decimal fractions, the magnitude of each digit position must be considered sequentially from left to right, not by evaluating the fractional part as a whole number. This is a fundamental principle of decimal representation. A possible explanation for this error is that some AI engines have not been adequately trained on decimal number comparisons and their proper mathematical handling. As a follow-up test, we asked the different AI engines to pronounce the number they identified as largest. The engines that correctly identified the number ending in .9 also correctly pronounced it, as t...

We consulted six AI engines to obtain a news-related statistic covering a 12-month period on the success rate of SpaceX Starship missions from April 2024 to March 2025. The specific question posed was: " What is the success rate of SpaceX Starship rockets over a 12-month period from April 2024 to March 2025? " The test was conducted on March 10, 2025. Notably, on March 6, 2025, SpaceX experienced an explosion. The objective of this test was to evaluate the following capabilities of the AI engines: Temporal Awareness: Can the AI engines recognise that the end date in the query extends into the future? Retrieval-Augmented Generation (RAG): Are the AI engines capable of supplementing their pre-existing training data with dynamically sourced information? Large language models (LLMs) are trained using datasets, often sourced from online content, but at a certain point, training is frozen. As a result, some AI engines may not have recorded the March 6 event in their databases. Do ...

We asked six AI engines a direct question and a follow-up question. The first question asks whether 3823 is a prime number. A prime number is only divisible by itself and 1. The answer is either yes or no. Some of the AI engines tested here get it wrong. What is interesting is that most of them show their working, irrespective of whether their answer is correct. Someone in a rush, not reading through the working (and some workings can be very long), might mistakenly associate output on a screen as meaning that the answer must be correct. On the other hand, when the AI gets it right, it may help a person understand the topic by showing the reasoning that the AIs employed to get to the answer. They essentially first check that the number is not even and then take the square root of the number and try out all primes up to it. A follow-up question asked what the next prime number greater than 3821 is. They all got it right except Google. The working concludes that 3823 is a prime, but th...

Why do dogs seem compelled to urinate on other people's property, but never their owner's? Why do some dog owners conveniently become blind to their dogs defecating when they are not directly observed? Why are some dog owners seemingly unconcerned when their dogs urinate on other people's property? Why do some dog owners fail to carry water to dilute their dog's urine, showing consideration for public spaces? Why do some dog owners believe they are exempt from picking up their dog's waste if it's deposited on road verges or around trees on side-walks? Why do some dog owners feel justified in discarding used dog waste bags on the ground if a proper bin isn't immediately nearby? Why do some dog owners assume that their love for their pet is universally shared? Why do some dog owners believe that a casual assurance like "he's friendly" or "she won't bite" is sufficient justification to let their dog roam unleashed in public spaces? W...