Which Table Format Do LLMs Understand Best?

Summary

When discussing the reliability of AI-based systems, there’s something fundamental that doesn’t get enough attention: what’s the best format for passing tables of data to an LLM? Should you use markdown tables or CSV? JSON or YAML? Or does some other format work better than any of these? Why This Question Matters As AI systems become integral to data analysis, business intelligence, and decision-making processes, understanding format sensitivity is crucial for: Data Pipeline Architecture: Structuring data workflows for maximum AI comprehension Performance Optimization: Reducing processing overhead while maintaining accuracy Cost Management: Minimizing token usage and API costs in production systems Many RAG pipelines involve ingesting documents that contain tables of data. If we’re not formatting that data in a way that it is easy for an LLM to consume, then we may be needlessly hurting the accuracy of the overall system. Our Methodology We designed a controlled experiment to test how the formatting of a set of data would affect how accurately an LLM could answer questions about that data. Our tests involved passing 1000 records to an LLM and asking it to answer a question based on the data. We then evaluated whether it answered correctly or not in each case. We repeated this process for 1000 questions, using each of 11 different data formats. Dataset: 1,000 synthetic employee records with 8 attributes each (ID, name, age, city, department, salary, experience, project count) Questions: 1,000 randomized queries about specific data points Model: GPT-4.1-nano Formats Tested: 11 different data representation formats Example Question-Answer Pairs Q. "How many years of experience does Grace X413 have? (Return just the number, e.g. '12'.)" A. "15" Q. "What is Alice W204's salary? (Return just the number, e.g. '85200'.)" A. "131370" Notes on Methodology We opted to pass a relatively large number of records to the LLM in order to test its limits. In practice, with a large st...