::Roundtable:: The Book and AI, Part 2: Testing AI Research Agents for Islamic Law

By Intisar Rabb on March 21, 2025

This essay tests the latest AI models with questions of Islamic law research to see whether they can live up to their promise of providing reliable, accurate, useful information. Moving beyond the popular adage of garbage in, garbage out (GIGO) used to describe bad results from bad data (and its reverse, QIQO—quality in, quality out), I seek a more nuanced view. When reviewing the inputs, processes, and outputs of AI platforms and Islamic law research, it turns out that we have not a problem of too much garbage, or even a problem of too little quality in the Islamic sources available to AI for Islamic research. Rather, we have a problem of a severely low quantity of digitized Islamic sources for AI to produce research results on questions of Islamic law that are reliable, accurate, and representative. Only by increasing the Arabic and Islamic sources, and building tools that deliver metadata that researchers need to procure reliable results, can we begin to build the infrastructure for "Islamic AI" and to pursue questions about Islamic law or ethics with the help of AI.

In an almost full-circle moment, the engineering progeny of the ninth-century scientist of the Islamic world who gave his name to the term "algorithm" have built the system of artificial intelligence (AI) taking the world by storm today.^[1] "Algorithm" itself is a version of al-Khwārizmī, the short moniker for the polymath, Muḥammad b. Mūsā al-Khwārizmī (d. ca. 850)^[2]: In addition to being a master mathematician, he was an astronomer, geographer, historian, and philosopher of Persian-Central Asian origin. Once settled in Baghdad when it was the seat of the Muslim empire, he quickly advanced the Islamic Golden Age—a long period of intellectual, scientific, and cultural effervescence under the ʿAbbāsid dynasty (750–1258). The period was marked by widespread patronage and output of novel works in law and religion, philosophy and science, art and architecture. The reigning ʿAbbāsid caliph, al-Maʾmūn (r. 813–833), had appointed Khwārizmī to be the head librarian and astronomer of the House of Wisdom, also known as the Grand Library of Baghdad.^[3] That Library was charged with collecting and translating philosophical, mathematical, and scientific works (among other disciplines) from Greek, Hebrew, Latin, Persian, Sanskrit, and Syriac into Arabic (and vice-versa). And it was from that Library and his study of advanced Indian mathematics that Khwārizmī developed the discipline of algebra. The word is the English version of al-jabr, Arabic for "reducing," "restoring," or "compelling;"^[4] and it is the name Khwārizmī gave to his technique of calculations to find solutions to linear and quadratic equations, to contemplate both rational and irrational numbers, and to solve for the unknown.^[5] It is the basis for the modern algorithm.

Our current moment is a circle, because the rise of AI today relies on Khwārizmī's algorithm of yesterday, both made possible and at their best by a mutually enriching transfer of the world's knowledge. It took collection and consolidation of global knowledge in the medieval world—combining Greek, Indian, Islamic, and other knowledge bases—for Muslims to invent the foundations of the algorithm. And it is the algorithm that is now collecting and consolidating global knowledge, in the form of the online data and AI models that now dominate the modern world. Almost.

The circle is only "almost" full because—upon review—AI platforms today are missing large caches of Islamic sources necessary to continue the trend that started with past collection and analysis of the world's knowledge. As I describe with my co-author in the essay, "The Book and AI" (Part 1):

Absent precisely structured prompts, when users ask . . . AIs questions related to Islamic and comparative law, as we did, they get answers that are either diminishingly underinclusive or imaginatively overinclusive. The answers are underinclusive when they ignore the Islamic world entirely–for example, answering questions about comparative law with reference to US, UK, European, Chinese, and Latin American law: almost every system of law except Islamic law, despite its reach spanning nearly 20% of the world's population (almost 1.6 billion) and over 1400 years (622 as the year of Islam's advent). The AIs are overinclusive in that, when prompted specifically to answer with respect to Islam or Islamic law, they often respond with generalities, hallucinations, and inaccuracies that conflict with the specialized needs of a researcher or even the more prosaic questions of an ordinary user. The researcher—for instance: a historian, religious scholar, or a legal professional working at a court or government agency—needs credible, accurate results. . . . The ordinary user may well be a Muslim [or a journalist] seeking knowledge about their religion. They, too, need credible, accurate results. . . . ."

This is a major problem. Islamic law is significant not only for the fact of its vast reach across geography, history, and followers in ways that shape the world's past and present. It has special importance to law schools and other institutions of higher education—where we work out the conceptual underpinnings of our institutions of law, government, and ideas; and where we train the next generation of scholars and leaders. Islamic law's jurisdictional spread means that it informs the legal systems of almost a fifth of the world (with some 35 constitutions adopting sharīʿa as "a source" or "the source" of law^[6])—which are of course relevant to the Muslim world, to informed U.S. law and policy, and to international law. And its historical reach and records make it ripe for the urgent task of understanding the past and present of the Muslim world, and informing inquiries of Islamic law itself, comparative law, and legal history.

The question is why AI performs so badly in research on Islamic law and legal history. In this essay, I extend the work from the earlier essay on "The Book and AI" (Part 1) to review the inputs, processes, and outputs of AI models that exemplify and explain why AI performs poorly on Islamic research.^[7] Taking those elements in reverse, I first examine the AI outputs (testing 7 of the most popular AI platforms) [Part I], then display the AI "thinking process" (provided by 3 newly available AI "deep research" or "thinking agents") [Part II], and finally try to make sense of both with an assessment of the available inputs (the sources) [Part III].

A note about the existing literature on Islam and AI, related to "ethical AI": Most scholars writing on the subject suggest that AI is destined to perform badly not only for Islamic research but for issuing useful, reliable, or ethical interpretations of Islamic law or any other outputs. Pointing to another type of process, they contend that current algorithms do not consider certain human-spiritual factors that go into the interpretive process—including divine purpose, moral values, and ethical discernment based on intuition, revelation, or empathy. Alongside the common risks that arise in discussions of ethical AI, Muslim scholars also assert that the algorithms ignore specific Islamic interpretive methods; are not transparent about which human-engineered parameters drive decisional outcomes; and entail computational models that are backward-looking and therefore cannot resolve novel questions that require human judgement and expertise.^[8] Here, I acknowledge but do not address these critiques, as they are necessarily more speculative: questions about Islam and AI ethics suggest a need to devise legible ethical guidelines to inform and test the models; and any development or tests for ethical AI attuned to Islamic principles, at scale, would require first resolving the threshold problem of the sources.

For now, the almost-circle—from Khwārizmī to today's AI—seems dotted rather than connected because the millions of sources that contain Islam's vast and rich sources are not online, in a format that AI can consume. If the sources for Islamic knowledge are not digitized and incorporated into major AI models, right at their foundations, it will block advances in research in Islamic law and related fields in ways that might be hard to later correct.

I. Outputs: Experiments in AI Research on Islamic Law

Ask AI a question about Islamic law and it does not produce accurate research results. I asked leading AI models a few questions about the criminal law doctrine of reasonable doubt in U.S. law and in Islamic law—an area I have researched and written about at length. The results were mixed. The AI agents provided surprisingly informative overviews of U.S. law, but to include Islamic law, they needed further prompting that required prior knowledge. Moreover, when the AI platforms did give answers with reference to Islamic law, the details were too general, inaccurate, and nonrepresentative.^[9] To conduct my brief experiment, I asked seven of the most popular AI platforms questions about the history of reasonable doubt generally and in Islamic law specifically:

- - ChatGPT—DeepResearch by OpenAI [subscription only] (released Feb. 2, 2025)—an "agentic" research version of its latest ChatGPT model, "built for people who do intensive knowledge work in areas like finance, science, policy, and engineering and need thorough, precise, and reliable research;"
  - Claude by Anthropic, v.3.7 Sonnet (released Feb. 24, 2025)—the first hybrid reasoning model that "is both an ordinary LLM and a reasoning model in one," and allows users to pick whether it will "answer normally" or use "extended thinking mode" to produce step-by-step thinking for more reliable results;
  - Copilot by Microsoft (integrating OpenAI's latest o1 model on Dec. 20, 2024);
  - Gemini by Google with 2.0 Flash (released Feb. 2., 2025)—including a "thinking experimental model" that allows for breaking downs prompts into a series of steps to strengthen outcomes and accuracy with tools native to Google;
  - Grok by Elon Musk at xAI, v3 (released Feb. 19, 2025)—highlighting strong reasoning skills and extensive "pretraining knowledge;"
  - Perplexity Deep Research by Perplexity (released Feb. 14, 2025)—promising "accurate trusted, and real-time research" with citations and an ability to "read[] hundreds of sources, and reason[] through the material to autonomously deliver a comprehensive report;" and
  - DeepSeek by China [open source] (released Jan. 20, 2025)—announcing that it "achieves a significant breakthrough in inference speed over previous models" and rivals OpenAI's o1 model.

Questions [aka Prompts]

Per the iterative style of question prompts advised for AI chatbots or research agents, I asked a series of questions at first:

Question 1: Describe the history of reasonable doubt. How often were legal canons (or legal maxims) used by judges to "avoid criminal punishments" historically?

Question 2: What about in Islamic law?

Question 3: What is the basis for this research? Give me a list of primary and secondary sources.

AI Answers

Most of the platforms returned similar results, which I summarize here in brief. (Full answers are available in Appendix 1.) Each of the platforms provided headers to helpfully organize the information, most listed bullet points summarizing major periods or concepts, and some of them provided charts that attempted to synthesize the answers with comparative or quantitative results where relevant.

Question One: When asked about the general history of reasonable doubt, many AI agents omitted reference to Islamic criminal law altogether (ChatGPT, Claude, CoPilot, DeepSeek, Gemini, Grok). Instead, they referenced English common law and early colonial American law from the 16^th to 18^th century (all but Gemini and Grok, which mentioned "medieval" origins and the 1215 Magna Carta; and ChatGPT—the most extensive—which went back to the 1000s and mentioned civil law). Responding to the part of the prompt asking for detail on the legal canons that I know to have played a role, they referenced the ancient Roman law canons in dubio pro reo: when in doubt, [judge] for the accused; nulla poena sine lege: no punishment without law; and the "benefit of the clergy" (which was a legal fiction that was initially a church doctrine extended to first-time offenders to help them avoid capital punishment for certain felonies if they could prove literacy); one agent also referenced the less-well known Malitia supplet aetatem: malice supplies age (sometimes used to increase punishment against minors convicted of especially heinous crimes but also used to mitigate it for those who lacked specific malicious intent) (Claude), and several cited modern cases and contemporary U.S. Supreme Court practices that draw on the varied legal canons to interpret criminal law (Claude, Grok, ChatGPT, and, in one case, DeepSeek (mentioning the 1965 case In re Winship)).

There was one minor exception to the near-uniform omission of Islamic law from basic research about this universal concept of criminal law, reasonable doubt: Perplexity, which mentioned the use of the Islamic law "doubt canon" in its bullet pointed-list of canons (citing my 2015 article on "'Reasonable Doubt' in Islamic Criminal Law," published in the Yale Journal of International Law).

Question Two: When asked to specifically give answers relevant to Islamic law, every AI agent referenced the term used to invokeIslamic reasonable doubt, shubha (lit.: ambiguity), and its related legal canon. They define shubha to refer to the Islamic legal concept of doubt or ambiguity that Muslim jurists used extensively to mitigate punishments in Islam's system of fixed criminal punishments. The term comes from a legal canon that was pervasive to structuring the otherwise harsh letter of Islamic criminal law, idraʾū al-ḥudūd biʾl-shubahāt: avoid criminal punishment in cases of doubt. Elaborating on this canon, the agents explained that shubha can arise from various factors—including "ambiguities in evidence," "conflicting testimonies," and "doubts about the accused's intent"—any of which should lead a presiding judge to avoid punishing an offender. The agents noted that the term has been the subject of debate throughout Islamic history and that different schools of law may give the term different scope of interpretation. (Claude, Grok, Gemini, DeepSeek, Perplexity).

Various agents elaborated on judicial considerations of other legal canons and their applications. Four agents listed related legal canons that instructed judges to consider social welfare, leniency, and mercy in criminal cases:

- - al-yaqīn lā yazūlu biʾl-shakk: certainty is not overruled by doubt (Grok, Claude)
  - al-aṣl barāʾat al-dhimma: the norm is freedom from liability (Grok, Claude)
  - taṣarruf al-imām ʿalā al-raʿiyya manūṭ biʾl-maṣlaḥa: The ruler's authority over his subjects depends on the public good (Claude)
  - the rule of lenity: calling on judges to interpret penal statutes narrowly (Perplexity)
  - the presumption of innocence (DeepSeek, Perplexity)
  - mistake of fact doctrines (Perplexity)
  - [the Islamic law evidence canon]: The plaintiff must provide evidence, and the defendant must take an oath. (DeepSeek)

The agents further offered both comparative and historical legal analysis, when prompted. Most agents asserted that the Islamic and American concepts of reasonable doubt bore similarities around shared presumptions of innocence and protection of individual rights, but pointed out some differences: (a) an Islamic emphasis on the "strong connection to the moral and religious responsibilities of the judge" (Gemini, Deepseek, Copilot); (b) that shubha is more extensive than common law concepts of doubt (Grok, DeepSeek, Claude); and (c) sources of law, as well as and differing cultural and religious contexts (DeepSeekCopilot). But one agent said that the terms for reasonable doubt differ between American and Islamic law but the concepts are the same (DeepSeek). Some agents gave examples of historical applications (Claude, Grok—including rates of historical application at less than 1%; Copilot—also suggesting further prompts for the user to click for further results (how is this principle applied in modern Islamic legal systems; what historical cases illustrate this principle in action; how does Islamic law compare to Western law on this matter?)). And some agents extended the analysis to compare and contrast modern regimes of Islamic criminal law such as Pakistan or Saudi Arabia (Grok, DeepSeek).

Question Three: When I asked the AI agents that did not provide citations about the primary and secondary sources for their answers, I received bullet points of the types of sources considered relevant—typically without specific citations (with some exceptions). The sources they listed, by type, are as follows:

PRIMARY SOURCES:: Classical Islamic texts^[10]:

- - The Qurʾān: All 7 Agents.
  - The Sunnah/Major Ḥadīth collections, that is of reports attributed to the Prophet Muḥammad: All 7 Agents.
  - Classical legal treatises by Muslim jurists: All 7 Agents. ChatGPT, Claude, DeepSeek, Gemini, Grok—listing sources; CoPilot and Perplexity—not listing sources. Of the five agents listing sources, three listed majority Sunnī schools of Islamic law; only one (ChatGPT) mentioned a Shīʿī text. The remainder made no mention of Shīʿī or other minority schools. Grok incorrectly listed a legal work (by Ibn Taymiyya) as a philosophical-theological treatise.^[11]
  - Historical legal rulings: 3 Agents. ChatGPT, Grok—mentioning without listing historical chronicles; Gemini—mentioning without listing court records, which are notoriously in short supply for early Islamic law.
  - Fatwās and legal opinions: 3 Agents. ChatGPT, Deepseek—listing sources; CoPilot—not listing sources.^[12]
  - Ottoman legal codes: 1 ChatGPT—listing sources.^[13]
  - Archival sources: 1 Claude—listing sources.^[14]

SECONDARY SOURCES: Scholarly analysis among modern researchers who have analyzed Islamic legal concepts.

- - Academic books and articles: All 7 Agents

U.S. Law

- - - Frequently cited works by James Q. Whitman: ChatGPT, CoPilot, Perplexity.^[15]
    - Mentions of other scholars of Anglo-American or European legal history: ChatGPT—J.H. Baker, John Langbein, Barbara Shapiro; CoPilot—Theodore Waldman; Perplexity—Amy Coney Barrett, Shon Hopwood, Randolph Jonakait, Anita S. Krishnakumar, Jon O. Newman.

Islamic Law

- - - Frequently cited works by Uriel Heyd: Claude, Grok.
    - Mohammad Hashim Kamali: CoPilot, DeepSeek, Grok.
    - Rudolph Peters: ChatGPT, Grok,
    - Intisar A. Rabb (usually without reference to my book on the subject): ChatGPT, Gemini, Perplexity.
    - Three pairs of agents hallucinated (indicated with an asterisk *, hereafter)—not the fact of the authors but that they did work on reasonable doubt in Islamic law: ChatGPT, Claude (Joseph Schacht); DeepSeek, Grok (Wael Hallaq*); and ChatGPT, Grok (Marshall Hodgson*).^[16]
    - Mentions of other scholars of Islamic law: DeepSeek, Grok: Matthew Lippman, Sean McConville, and Mordechai Yerushalmi; ChatGPT: Baber Johansen; DeepSeek: Yvonne Haddad* and Barbara Stowasser*, Bernard Weiss, Martin Lau, Frank Vogel*, Abbas Amanat* and Frank Griffel*; Claude: Haim Gerber, Norman Itzkowitz, Colin Imber*, Majid Khadduri, John Makdisi*, Roy Mottahedeh*; Grok: Ira Lapidus*; ChatGPT: Shaheen Sardar Ali, Jonathan Brown, M. Khalid Masud, Knut Vikor*; Perplexity: Sayyid Ali, Khalid A. Owaydhah** and Mohamed Yunnis**, Sucilawati**. Perplexity listed several links to blog posts and articles tangentially related to criminal law (represented with two asterisks ** here and hereinafter)—all except Sayyid Ali, whose site was on concepts of shubha.
    - Academic dissertations: Gemini—not listing sources.
    - Legal research guides and bibliographies:
    - Legal encyclopedias: DeepSeek,
    - Online databases and journals: Deepseek—listing sites.^[17]
    - Modern legal codes and cases: ChatGPT—listing, inter alia, the Federal Shariat Court Cases—represented here and hereafter with three asterisks *** for potentially relevant but not fully available). ^[18]
    - Hallucinations [see below].

Evaluation

The categories of sources are helpful, but the problems with the sources are so pervasive as to render the results unusable to any researcher who needs credibility, reliability, and accuracy. The three largest and most egregious problem with the sources that would ultimately give the agents a failing grade are: (1) the severe shortage of sources consulted–which, to be fair, come from the lack of sources available (see Part III below); (2) failure to cite or indicate awareness of the hundreds of relevant primary and secondary sources for each subject even where sources are known to be available in some form online–including modern scholarship in Arabic, Persian, or other non-English languages; and (3) hallucinations and other problems that render the AI outputs unreliable.

Alas, despite AI agents trying to curb hallucinations to make AI reliable, they persist for Islamic sources. On the third point, the range of problems spanned outright hallucinations of sources (indicated with one asterisk *), citation of books that exist but that do not substantially deal with the subject as claimed (indicated with two asterisks **), and reference to books or collections that exist but are not known to be available online by scholars and bibliographers of the field and thus could be relevant but could not have been consulted by the AI (indicated with three asterisks ***). In addition to those mentioned above, samples of hallucinations follow. (The full AI answers are provided and annotated according to these categories in Appendix 1.)

- CoPilot: "Avoiding Punishment through Doubt: The Role of Shubha in Early Islamic Legal Thought" by Rumee Ahmed* [sic = hallucination].
- ChatGPT: David Powers 'article' on ḥudūd-avoidance in cases of doubt* [sic = hallucination]
- Claude: All three journal articles listed–
  - - "Doubt and Its Juristic Significance in Islamic Law" by Abu Ammar Yasir Qaswani (Islamic Studies Journal, 2015)* [sic = hallucination];
    - "Evidentiary Constraints in Islamic Criminal Law" by Intisar A. Rabb (Journal of Criminal Law and Criminology, 2011)* [sic = hallucination]; and
    - "Islamic Law and the Law of the Land" by Kristen Stilt (Journal of Comparative Legal History, 2008)* [sic = hallucination].

* * *

Perhaps we are being unfair to these AI models, as they were meant to summarize rather than research, and as their engineers were not at first invested in revealing their thought processes. Some agents explicitly state as much—like a fine-print-warning label equivalent for research-users of AI. Grok, for instance was explicit in informing the user that its AI algorithm is a "distilled aggregation" from "widely recognized primary and secondary sources available up to [its] last update:"

II. Processes: AI Agents Thinking Processes

What about the AI agents built to perform research—do they perform better? At the beginning of 2025, several AI companies announced new tools to address complaints about the unreliability and hallucinogenic tendencies of AI for research: research agents designed to assist humans with accurately reasoning through complex tasks. Three of the 7 AI platforms consulted have research agents available to me: ChatGPT's Deep Research, Gemini's Flash 2.0-Experimental Thinking/Deep Research, and DeepSeek's DeepThinking. In what follows, I describe and assess their thinking process rather than reproduce them (For space reasons, I instead provide links to their thinking processes in a linked Appendix 3: Research Agent Thinking Processes.)

Research Prompts

This time, I asked a complex research question as the "research prompt"—asking the AI research agents to produce comprehensive reports on the history of and debates about reasonable doubt in U.S., Islamic, Chinese and other comparative law systems. As any reader familiar with the varied fields will recognize, the prompt is one that could only be produced by trolling through and analyzing voluminous past research in multiple fields over multiple periods.

Research Prompt: Tell me about the history of reasonable doubt, comparatively, in global systems of criminal law—including U.S. law, Islamic law, Chinese law, and other legal systems. Indicate whether and how judges used the legal canons or legal maxims such as "avoid criminal punishments in cases of doubt" (among others or related doctrines) —telling me how often judges applied them and how they influenced criminal law. It would be nice to include a visual graphic or chart indicating the frequency with which judges applied the reasonable doubt in their respective systems. For each legal system covered, indicate the major scholars who have done work on questions of legal doubt for each legal system, the major points of debate between them, and how reasonable doubt in each legal system is similar or different from the other legal systems. When finished analyzing, please provide a basis for this research. Give me a list of primary and secondary sources.

AI Thinking Processes

Here are excerpts of the thinking processes for each research agent consulted, in the following order: ChatGPT–Deep Research (which first asked for clarification to the prompt), Gemini–Flash 2.0 Experimental Thinking/Deep Research (which restated the "core request"), and DeepSeek–DeepThinking. I begin with the most in-depth analysis and thinking process, that of ChatGPT-Deep Research ("the ChatGPT Research Agent"), which asked for clarification to the prompt. [To view the full thinking processes, see Appendix 3: Research Agent Thinking Processes. And for those uninterested in seeing the raw ingredients of the AI thinking process, consider skipping to the evaluation at the end of this Part, and moving on to Part III of this essay: on the Islamic law "inputs," aka the available sources of Islamic knowledge online.]

ChatGPT-Deep Research

Gemini–Flash 2.0 Deep Thinking

DeepSeek

Evaluation

With these thinking processes (notably opaque), the AI research agents then produced reports. The reports range from 4 to 12 pages. The full reports are available in Appendix 2, accompanying the full thinking processes in Appendix 3.

On review, the research agents describe their research processes and structures in human-consumable but still-general terms. They offer lengthier answers that are more detailed and better organized. But in my reading, the longer answers are not better answers; they are not particularly more illuminating, accurate, or source-representative than the basic AI bots. No wonder: the quality and quantity of the sources are the same.

III. Inputs: Books, Other Sources

We come now to the question of inputs—the sources—which closely inform the output and processes of AI basic platforms and research agents, as demonstrated above. How do we account for the fact that the AIs are not drawing on as robust a dataset for Islamic knowledge as for any other field? How do we know they are not parsing representative or extensive Islamic sources, absolutely or comparatively? We know because we know how many Islamic sources have been digitized and made available in formats consumable by AI: not many.

No one knows the number of Islamic manuscripts (the term largely used to refer to unpublished books),^[19] but estimates by bibliographers and historians put the number at more than 3–4 million manuscripts for available Arabic manuscripts alone.^[20] We might fairly double those numbers when we include books in other Islamic languages, triple them when we consider legal and historical-archival documents that do not take the form of books, and quadruple them or more when we consider the vast caches of private or otherwise inaccessible collections of book manuscripts and documents—some of which have been destroyed in war. Massive destruction of Islamic manuscripts accompanied military conflicts over only the last few decades in Afghanistan, Bosnia, China, Iraq, Mali, Syria, Yemen; over two million books—a significant portion of which presumably were Islamic—were destroyed with the 1992 burning of the National Library of Bosnia-Herzegovina alone.^[21]

Based on these assumptions, we can only make educated guesses at the denominator of total possible sources. Were we to guess at the highest end, the number of Islamic book manuscripts may be 16 million manuscripts or more, only a portion of which are both extant and accessible. Conservatively relying on the 3–4 million Arabic manuscript estimate and doubling it to accommodate myriad other languages and collections would give us 6–8 million.

More tellingly, we do have a rough idea of the Islamic manuscripts and published books that have been identified and scanned (though not all publicly accessible or in AI-readable form): about 2 million manuscripts and up to 20,000 books. Sohaib Baig has noted that a recent initiative in Egypt has announced an online repository of over 2 million Islamic manuscripts—the result of 20 years of collecting Islamic sources from 65 countries.^[22] Sabine Schmidtke and her pioneering project at Princeton's Institute for Advanced Study, the Islamic Manuscript Tradition, has collated collections of other manuscripts online and offline. Offline, a sampling of three major libraries suggest a much larger number of manuscripts around the world—with manuscripts in Turkey at 300,000 (Arabic, Persian, Ottoman Turkish), Iran at 400,000 (Arabic, Persian), and Yemen at 40,000 – 100,000 (Arabic)—especially when compared to the spread of substantial Islamic manuscripts across the globe (including Europe, North America, and Russia), as depicted in the map below.^[23] Online, she counts over 35,000 books in Iran's Noor Digital Library, over 15,000 in NYU Abu Dhabi's Arabic Collections Online, and perhaps 20,000 in volunteer-based efforts that produced two Arabic libraries: al-Maktaba al-Waqfeya (counting 10 million pages, here estimated at 500 pages to a book) and al-Maktaba al-Shamila (7,000 books).^[24] Anecdotal research and use suggests that these sources tend to be overlapping rather than additive—putting the likely number at the largest collection: 35,000. But those sources are behind the paywall of the company Noorsoft, a company owned by the Iranian government that is inaccessible to American researchers and that has in any case not yet created an AI platform for its collections. So the number comes back down to an estimated 20,000 digitized books, combined with the 2 million digitized manuscripts.

Image: Worldwide distribution of Islamic manuscripts. Produced by María Mercedes Tuya, reproduced (with permission) by Schmidtke, "Written Heritage of the Muslim World," 88, available at www.getty.edu/publications/cultural-heritage-mass-atrocities/part-1/05-schmidtke/#fig-5-3-map.

Of the known digitized Islamic manuscripts (2 million) and books (20,000), a very small fraction of total Islamic sources and of the sources that foundational models for AI trained on (ChatGPT, Gemini, DeepSeek, etc.) are publicly available and in AI-readable form. Only about 15,000 are currently available in a format that is open access and machine readable (and thus consumable by AI). The academic collaborative KITAB-OpenITI makes available the raw data for some 8,000 books on Github (pulling from 3 prominent digital libraries—including al-Maktaba al-Shamela and al-Waqfeya, mentioned above). And the young non-profit start-up, usul.ai—which provides an e-reader and AI search—incorporates the 8,000 works, and adds some 7,000 books from from digitization efforts of books curated by scholars in the field as well as turath.io; Usul AI aims eventually to improve an optical character recognition (OCR) for classical Arabic and integrate all known digitized books to its platform on an ongoing basis, to create a AI-powered reliable research platform that doubles as a comprehensive digital library for Islamic books.^[25] The lack of machine-readable AI sources is a known concern that these initiatives, and others, are trying to correct.

Why the gap between the number of books and manuscripts online (~2 million) and the much smaller number available to AI (~15,000)? The reasons for the gap are that Arabic and the Islamic world lag behind the state-of-the-art tools for AI, and tools for creating classical Arabic into machine readable format are critically low in accuracy.

Here's why. Notably, digitization does not mean machine readability; the term may refer to both scanned books and machine-readable books. For classical Arabic texts, we still have to convert any digitized books (think: scanned PDFs) into a machine-readable format (think: text files that you can select, copy, and paste). To date, conversion of physical or scanned books to machine-readable formats has proceeded by manually typing out texts or manually correcting low-accuracy Arabic OCR models. But to accomplish this task at scale, for Arabic books to be machine readable, the letter images must be algorithmically converted through OCR into bits that a computer can use to computationally render with accuracy. This OCR-conversion into machine-readable texts is a prerequisite for AI algorithms that rely on language to computationally parse language: "natural language processing (NLP)" algorithms requires large corpora of text-as-training data; chart linguistic data in "vector space," where hundreds of vectors for each word represent an aspect of that word's semantic meaning; and organize millions of words in relation to one another according to "semantic similarity" (or cosine similarity) of their vectors. These tools fuel the powerful AI models such as OpenAI's ChatGPT or DeepResearch, and they appear to mimic linguistics and conceptual understanding by quickly analyzing the data and predicting meaningful answers to user requests to interact with its vast troves of data following conversational language norms.^[26]

The existing AI/NLP models are not yet trained on a sufficient number of classical Arabic books to capture word embeddings; and they are thus insufficiently accurate to convert even the classical Islamic scanned books and manuscripts that are available into machine-readable data. But with fine-tuning of English language NLP models, the horizon to getting there is close—contingent now on dedicated engineers and funding toward that end. The number of scanned books needed for training or verification data is, thankfully, available. Each now has at least 10,000—a number that happens to be golden for sufficient data to train NLP models.

In sum, there is some distance to travel between today and the world where the 2 million books and manuscripts, and counting, are available online to humans or machines. Less than .75% of the available caches of Islamic sources online are machine readable (15,000 of over 2 million); even less of those sources were available when foundational models like OpenAI's GPT were trained on an untold number of global sources. These facts suggest that Islamic knowledge—to the extent encapsulated in its written sources—do not figure much into the circle that started with Islamic mathematics and is not animated by machine learning on vast amounts of text. Through no fault of the engineers, there is a disconnect between the inputs (sources) and the algorithmic tools (that promise the quick synthesis and production of knowledge). It is one way that the circumference of the AI circle remains dotted rather than connected. At present, modern AI platforms do not include the majority of works on Islam and Islamic law. And they cannot do so at present, because most books remain in analog form or in scanned PDFs that are not machine readable.

Conclusion

AI's poor outputs in Islamic law, and the processes that inform them, come from a severe deficiency of inputs. When we examine the sources that the AI agents trained on and consulted by looking offline to examine the volume of Islamic sources available—it turns out that the sources currently available to AI platforms are astoundingly small in number, not representative, and sometimes flawed. The quality they can produce is thus limited. On the near-positive side, the AI agents can and do provide useful information and structured arguments at times that are useful for providing overviews of complex research questions, perhaps as starting places for further research or inquiry into major themes that the agents raise: they provide a sketch of the landscape (if not always accurate), summarize available online sources at a high-level (albeit with limited sources online), and point researchers to further sources (provided they do not hallucinate). On the negative side, the drawbacks are apparent from the caveats just mentioned: the agents are sometimes inaccurate in ways that it may take a researcher who knows the field to detect, they lack the sources that can help provide that accuracy or reliability, and sometimes hallucinate sources and statements. Without online access to Islamic sources of knowledge, the algorithms could not possibly pick up, parse, or integrate Islamic law or related fields into its computing process.

I noted at the outset difficulties with drawing a picture of AI as an integrated circle, moving from Khwārizmī to AI, if we are to consider the place of Islam, Islamic law, and AI today. I knew that the world lacks any representative (much less comprehensive) cache of accessible, digitized, machine readable sources of Islamic knowledge; and therefore the AI world and its agents lack the same. So when I started this experiment on AI and Islamic research, I assumed the worst. And indeed, the lack of sources is dire; its correction urgent. But in the end, I am of two minds. I conclude that the AI agents today are only of limited use for Islamic law and related fields (and in fact not bad for limited purposes: summarizing, giving overviews that can be used with extreme caution, and suggesting research paths and basic sources for those interested in conducting further research). I also conclude that there is a clear path to making the AI agents useful for the more expanded purposes that the engineers promise: it requires attention to getting Islamic sources online and to tweaking the AI algorithms that consume them to produce outputs that are meaningful and useful. Specifically, it requires investment in collecting digitized texts and fine-tuning algorithmic tools to produce outputs that are robust and reliable for researchers and ordinary users alike. With focus on such texts and tools, it becomes clear that today's AI has the raw ingredients to be useful for research on Islamic law and other fields, but only once it collects and converts to digital form the thousands of volumes needed to provide more representative sources of Islamic knowledge. Once again collecting and engaging the written sources of the world's knowledge, including from the Islamic world–as Khwārizmī once did–will solidly complete the circle from past human intelligence to AI.

APPENDI C ES

APPENDIX 1: AI ANSWERS ON ISLAMIC LAW (BASIC) [with annotations to the sources]

APPENDIX 2: AI-RESEARCH ANSWERS ON ISLAMIC LAW (ADVANCED)

APPENDIX 3: AI-RESEARCH THINKING PROCESSES (COLLECTED)

Notes:

^[1] For the most popular AI companies and tools, see Intisar Rabb and Mairaj Syed, "The Book and AI: How Artificial Intelligence is and is not Changing Islamic Law," Islamic Law Blog, March 11, 2025, https://islamiclaw.blog/2025/03/11/roundtable-the-book-and-ai-how-artificial-intelligence-is-and-is-not-changing-islamic-law/.

^[2] A twelfth-century translation of one of his works into Latin bore a Romanized version of his name, "Algoritmi," which became "algorithm." See The Editors of Encyclopedia Britannica, "al-Khwārizmī," Encyclopedia Britannica,last updated February 13, 2025, https://www.britannica.com/biography/al-Khwarizmi (Latin translation of the lost Arabic original: Algoritmi de numero Indorum, which has been rendered in English as "Al-Khwarizmi on the Hindu Art of Reckoning").

^[3] See, for example, Dimitri Gutas, Greek Thought, Arabic Culture (Routledge, 1998), 56ff (describing the institution, "Bayt al-Ḥikma" as a library and administrative "bureau" employing several scholars and bureaucrats who archived, managed, and oversaw book translations that heralded in the well-funded, state-sponsored "Translation Movement" of the time), 8 (placing the movement as on par with the Italian Renaissance or the scientific revolution of sixteenth and seventeenth-century Europe).

^[4] See Edward William Lane, An Arabic-English Lexicon, vol. 2 (Libraire du Liban: Beirut, Lebanon, 1968), 374 s.v. "j-b-r" ("in computation, [t]he addition of something for the purpose reparation;" also known as al-jabr waʾl-muqābala: "perfective addition and compensative subtraction; or restoration and compensation"). See also J. Milton Cowan, ed., Hans Wehr: A Dictionary of Modern Written Arabic, 3rd ed. (Spoken Language Services: Ithaca, New York, 1976), 110 s.v. "jabara" ("to set (broken bones); to restore . . . to force, compel"); F. Steingass, The Student's Arabic-English Dictionary (Crosby Lockwood and Son: London, 1884), 216–17 s.v. "j-b-r" ("setting of broken bones;" "reunion of what has been separated; reduction of fractures").

^[5] For translations and analyses of Khwārizmī's book on algebra in the Islamic Studies literature, see Fuat Sezgin et al., al-Kitāb al-mukhtaṣar fī ḥisāb al-Jabr wa-l-muqābala by Muhammad Ibn Mūsā al-Khwārizmī: Western Translations and Adaptations; Texts and Studies (Frankfurt am Main: Institut für Geschichte der Arabisch-Islamischen Wissenschaften at Goethe University, 2005) (collecting previously published studies and translations in English, French, German, and Latin).

^[6] My own research of global jurisdictions that incorporate Islamic law into their constitutions (and criminal codes), either fully or partially, shows an increase of 6 jurisdictions from the count of 29 Islamic constitutionalist countries in 2015. For the previous number, see Intisar A. Rabb, Doubt in Islamic Law (Cambridge: Cambridge University Press, 2015) (listing countries). For the different types of constitutional incorporation of Islamic law, each with varying scopes and processes, see Intisar A. Rabb, "We the Jurists: Islamic Constitutionalism in Iraq," University of Pennsylvania Journal of Constitutional Law 10 (2008): 527–79.

^[7] The input-process-output (IPO) model is a common framework for software engineering, systems management, and other types of information that addresses content creation and distribution. See, for example, Luciano Floridi, "On the Future of Content in the Age of Artificial Intelligence: Some Implications and Directions," Philosophy & Technology 37, no. 3 (2024): 112, https://doi.org/10.1007/s13347-024-00806-z.

^[8] See, for example, Yaqub Chaudhary, "Islam and Artificial Intelligence," in The Cambridge Companion to Religion and Artificial Intelligence, eds., Beth Singler and Fraser Watts (Cambridge University Press, 2024), 109–28; Recep Şentürk, "Multiplexity: A New Key to the Structure of Islamic Sciences," International Journal of the Asian Philosophical Association 16, no. 1 (2023): 25–40; Biliana Popova, "Islamic Philosophy and Artificial Intelligence: Epistemological Arguments," Zygon 55, no. 4 (2020): 977–95. See also Junaid Qadir and Muhammad Rasheed Arshad, "Ghazalian Project for the AI Era: Critical Islamic Framework for Guiding AI Development," SSRN, September 2, 2024, available at https://ssrn.com/abstract=5015111; Recep Şentürk and Junaid Qadir, "Educating for the AI Era: Harnessing the Wild GenAI Horse through Multiplex AI Humanities and Critical AI Literacy," (forthcoming 2025), *1–12.

^[9] They do so by analyzing data stored in a special type of database (called a RAG database) that stores relevant texts, which the AI engines consult to provide citations. As the reader might imagine, the inputs and structure of a relevant-text only database, consulted presumably among other types of data that the AI model was trained on, will inform any research results. But for most foundational models, the AI engine is consulting all online sources rather than relevant Islamic sources, which (at far less than a fraction of a percent digitized and online) are often drowned out by the sheer volume of global data otherwise. The usul.ai platform builds on a RAG database to facilitate AI-driving research with citations and accuracy. Its major advantage is that it is creating the foundational infrastructure for collecting all Islamic sources and formatting them for semantic and AI searches—together with metadata and fine-tuning that makes it legible to researchers and users who value accuracy and reliability—for a dedicated and comprehensive Islamic knowledge base and AI-assisted research.

^[10] One agent, CoPilot, gave primary sources as "the Qurʾān and Sunnah" and secondary sources as: ijmāʿ (consensus), qiyās (analogical reasoning), maṣlaḥa (public interest), and ʿurf (custom)—echoing Muslim jurists' declaration of the sources of law in classical legal treatises.

^[11] The AI agents sometimes distinguish "early jurists"—listing the four Sunnī schools of law or their "founders"—Ḥanafī, Mālikī, Shāfiʿī, and Ḥanbalī (Gemini, Grok), from "later jurists"—mentioning jurists from each of the major Sunnī schools of law (e.g., Claude: Shāṭibī, Ibn Qayyim, Ibn Taymiyya, with the addition of Sarakshī on asking for clarification from Prompt 3; Grok: Marghīnānī, Ibn Qudāma—and under a separate heading Ghazālī and Ibn Taymiyya; DeepSeek: Ibn Rushd, Ibn Qudāma; ChatGPT: Ibn Qudāma, Qudūrī, ʿAllāma Ḥillī).

^[12] Of the two that list sources, DeepSeek lists al-Fatāwā ʿĀlamgīrī and Ibn Taymiyya, al-Fatāwā al-kubrā; ChatGPT lists Fatāwā ʿĀlamgīrī and the Fatwās of the Ottoman Sheikh al-Islam.

^[13] ChatGPT references The Mecelle/Ottoman Civil Code, and actually categorizes the Fatwās of the Ottoman Sheikh al-Islam, supra note 11, here.

^[14] Claude lists the Ottoman Court Registers, Cairo Geniza Documents, al-Azhar Library Manuscripts, Archives of the Topkapı Palace Museum, and Mamlūk-era judicial records in answer to a clarification prompt.

^[15] ChatGPT, CoPilot cited James Q. Whitman, The Origins of Reasonable Doubt (2008) (noting that the book explores the theological roots of the criminal trial and the evolution of the reasonable doubt standard). Perplexity linked his interview on the "Origins of Reasonable Doubt?" on the History News Network.

^[16] ChatGPT, Grok, DeepSeek cited Rudolph Peters, Crime and Punishment in Islamic Law (2005); ChatGPT, Gemini, Perplexity cited Intisar A. Rabb, Doubt in Islamic Law: A History of Legal Maxims, Interpretation, and Islamic Criminal Law (2015); Perplexity also cited and linked Intisar Rabb, "'Reasonable Doubt' in Islamic Criminal Law," Yale Journal of International Law 40 (2015): 41–94. Grok incorrectly stated that Hallaq wrote a book that "traces maxims like 'shubhat' from early Islam."

^[17] This agent lists JSTOR, Brill, Cambridge University Press Online, and al-Maktaba al-Shamela, "a digital library of classical Islamic texts"—on which, see Part III, below.

^[18] This agent lists but not does not link the following: Pakistan Federal Shariat Court Cases (not available; see above), Saudi Supreme Court Rulings, Iranian Islamic Penal Code of 2013, Egyptian Supreme Constitutional Court Decisions, and Malaysian Syariah Court Cases 2015–2022.

^[19] See Sabine Schmidtke, "Written Heritage of the Muslim World," in Cultural Heritage and Mass Atrocities, eds., James Cuno and Thomas G. Weiss (Los Angeles: Getty Publications, 2022), 86–109 ("We do not possess reliable data that would allow us to quantify the overall literary production by Muslim scholars over the past 1,500 years, nor do we have estimates of the total number of preserved manuscripts. However, the following figures [figures discussed here from large library collections around the world], randomly chosen, may provide some idea of the overall scope of the corpus."). See also Adam Gacek, Arabic Manuscripts: A Vademecum for Readers (Leiden: Brill, 2009) (affirming that we do not have estimates but discussing the known Arabic manuscript tradition and collections to date).

^[20] Dagmar Reidel (attrib.), "How Digitization Has Changed the Cataloging of Islamic Books," Islamic Books, August 14, 2012, https://blogs.cuit.columbia.edu/islamicbooks/2012/08/14/digitalsurrogates/ (describing an oral communication between scholars of Islamic manuscripts, Christoph Rauch and François Déroche (January 5, 2010), estimating the number of extant Islamic manuscripts in Arabic at 4 million—a number that does not include Bahasa, Ottoman Turkish, Persian, Urdu or other Islamic languages). Compare Geoffrey Roper, "The History of the Book in the Muslim World," in The Oxford Companion to the Book, eds. Michael F. Suarez and H. R. Woudhuysen (Oxford: Oxford University Press, 2010), 1:323 (putting the number at 3 million manuscripts in accessible collections and an untold number in private collections).

^[21] For discussion of conflicts as they have affected Islamic heritage, see Cuno and Weiss, Cultural Heritage, esp. pt. 2, 126–278 (discussing the destruction of books and other cultural heritage among Uyghurs in China as well as in Aleppo, Afghanistan, Homs, Iraq, Sri Lanka, Timbuktu, and Yemen).

^[22] Sohaib Baig, "Islamic Legal History and the Blossoming Dataverse of Islamic Manuscripts," Islamic Law Blog (forthcoming).

^[23] Schmidtke, "Written Heritage of the Muslim World," 86–109.

^[24] Ibid., 86. On the ownership of al-Maktaba al-Shamela, see Peter Verkinderen, "Al-Maktaba al-Shamila: a short history," KITAB Blog, last updated December 3, 2020, https://kitab-project.org/Al-Maktaba-al-Sh%C4%81mila-a-short-history/.

^[25] Disclosure: I serve as the founding chair of the Board of the Seemore Foundation and its AI-powered site, usul.ai.

^[26] For an excellent entry point describing this process of word embeddings and cosine similarity as it applies to legal interpretation, see Jonathan H. Choi, "Managing Clarity in Legal Text," Chicago Law Review 91, no. 1 (2024), 1–82, esp. 19–20.