{"id":3860,"date":"2026-06-22T03:17:45","date_gmt":"2026-06-22T01:17:45","guid":{"rendered":"https:\/\/lecture.jose-marcio.org\/blog\/?p=3860"},"modified":"2026-06-22T03:23:55","modified_gmt":"2026-06-22T01:23:55","slug":"mainzer-kahl-limits-of-ai","status":"publish","type":"post","link":"https:\/\/lecture.jose-marcio.org\/blog\/2026\/06\/22\/mainzer-kahl-limits-of-ai\/","title":{"rendered":"Mainzer Kahle-Limits of AI – Theoretical, Practical, Ethical"},"content":{"rendered":"

\"\"<\/p>\n

Livre avec un contenu tr\u00e8s int\u00e9ressant, mais…<\/p>\n

Le livre est divis\u00e9 en 5 parties :
\n* Le concept d’intelligence artificielle\u202f;
\n* Limitations pratiques\u202f;
\n* Limitations th\u00e9oriques\u202f;
\n* Limitations conceptuelles\u202f;
\n* Perspectives pour l’IA hybride.<\/p>\n

Parlons des points n\u00e9gatifs avant les bons points.<\/p>\n

Ce livre, lu en anglais, est une traduction de l’original en allemand. Sorti en 2024.<\/p>\n

Deux probl\u00e8mes concernent la traduction. Le premier est l’apparition de plusieurs mots mal orthographi\u00e9s. Par exemple : \u00ab\u00a0programmation\u00a0\u00bb au lieu de \u00ab\u00a0programming\u00a0\u00bb or \u00ab\u00a0programme\u00a0\u00bb au lieu de \u00ab\u00a0program\u00a0\u00bb, \u00ab\u00a0Baysian\u00a0\u00bb au lieu de \u00ab\u00a0Bayesian\u00a0\u00bb, ou \u00ab\u00a0strange deviation\u00a0\u00bb au lieu de \u00ab\u00a0standard deviation\u00a0\u00bb. Ce genre d’erreur appara\u00eet de tr\u00e8s nombreuses fois. Ces erreurs auraient pu \u00eatre facilement d\u00e9tect\u00e9es par un v\u00e9rificateur d’orthographe.<\/p>\n

Encore pire, deux exemples de paragraphes montrent l’absence de relecture (il y en a d’autres du m\u00eame genre) :<\/p>\n

In general, we imagine a spatial system of identical elements
\n(\u2018cells\u2019) that interact with each other in different ways (e.g.
\nphysically, physically, physically, physically, physically)<\/p>\n

Corresponding proofs are mathematically possible if one
\naccepts the concept of the computability of the computability
\nof natural (and rational) numbers to real numbers.<\/p>\n

Un autre probl\u00e8me de traduction concerne les chapitres 3 et 4 et part du 2. Ce sont des th\u00e8mes tr\u00e8s ardus m\u00eame pour les initi\u00e9s (philosophie, statistiques, machine de Turing, Th\u00e9or\u00e8me de Godel…). Un initi\u00e9 comprendra rapidement que \u00e7a a \u00e9t\u00e9 traduit par quelqu’un qui conna\u00eet peu ou pas le sujet. La lecture devient p\u00e9nible.<\/p>\n

On peut difficilement jeter l’opprobre sur l’auteur original puisqu’ils sont connus comme \u00e9tant des connaisseurs de la sp\u00e9cialit\u00e9.<\/p>\n

Finalement, ces parties sont les plus int\u00e9ressantes parce qu’elles n’\u00e9voluent que tr\u00e8s lentement et ne changeront pas pendant plusieurs d\u00e9cennies. Par contre, la partie finale, perspectives sont tr\u00e8s li\u00e9es \u00e0 la technologie et \u00e9voluent tr\u00e8s rapidement.<\/p>\n

La partie finale parle des ordinateurs quantiques et des IA hybrides. Tout \u00e7a n’existe pas encore. Et, d’autre part, parle de ChatGPT qui \u00e9tait, \u00e0 l’\u00e9poque, \u00e0 sa version 3.5. On est maintenant \u00e0 la version 5.5. Il a beaucoup \u00e9volu\u00e9 depuis. Donc, C’est un peu obsol\u00e8te en ce qui concerne ChatGPT.<\/p>\n

M\u00eame si les parties conceptuelles \u00e9taient celles qui m’int\u00e9ressaient le plus, et qui m’ont laiss\u00e9 sur ma fait, \u00e7a reste un bouquin int\u00e9ressant. La partie \u00e9thique est trait\u00e9e rapidement, il y a plus \u00e0 dire. Mais on peut trouver mieux.<\/p>\n

Mon opinion sur l’IA ? Malgr\u00e9 l’accumulation de connaissances encyclop\u00e9diques que nous ne pouvons pas en avoir, je pense que l’IA ne remplacera jamais l’humain, ce n’est que par notre vie v\u00e9cue, l’enfance, les amiti\u00e9s, la conscience… Il y a beaucoup de th\u00e8mes, surtout \u00e9thiques, \u00e0 traiter encore.<\/p>\n

Dialogue avec ChatGPT<\/h2>\n

J’ai pos\u00e9 quelques questions \u00e0 ChatGPT au sujet de la premi\u00e8re citation ci-dessous.<\/p>\n

Q : What do you mean with \u00ab\u00a0As an AI language model, I don\u2019t have personal experiences, beliefs, or a subjective understanding of the world.\u00a0\u00bb<\/strong><\/p>\n

When I say:<\/p>\n

\u00ab\u00a0As an AI language model, I don’t have personal experiences, beliefs, or a subjective understanding of the world.\u00a0\u00bb<\/p><\/blockquote>\n

I mean several distinct things.<\/p>\n

I don’t have personal experiences<\/h4>\n

Humans learn not only from text but from living:<\/p>\n

    \n
  • seeing colors<\/li>\n
  • feeling pain<\/li>\n
  • eating food<\/li>\n
  • having conversations<\/li>\n
  • growing up in a culture<\/li>\n<\/ul>\n

    I don’t do any of those things.<\/p>\n

    When I describe Paris, I have never visited it. When I describe coffee, I have never tasted it. I learned about them from patterns in data, not from experience.<\/p>\n

    I don’t have beliefs<\/p>\n

    A belief is something an agent holds to be true and can revise based on evidence.<\/p>\n

    For example, you might believe:<\/p>\n

      \n
    • \u00ab\u00a0It will rain tomorrow.\u00a0\u00bb<\/li>\n
    • \u00ab\u00a0Democracy is the best political system.\u00a0\u00bb<\/li>\n<\/ul>\n

      I don’t actually hold such beliefs. I generate text that reflects information, arguments, and probabilities learned during training.<\/p>\n

      When I say:<\/p>\n

      \u00ab\u00a0The Earth orbits the Sun.\u00a0\u00bb<\/p><\/blockquote>\n

      I am not expressing a belief. I am producing the answer that best matches the available evidence and consensus.<\/p>\n

      I don’t have subjective understanding<\/h4>\n

      Humans experience the world from a first-person perspective.<\/p>\n

      You know:<\/p>\n

        \n
      • what red looks like<\/li>\n
      • what sadness feels like<\/li>\n
      • what it is like to be tired<\/li>\n<\/ul>\n

        Philosophers sometimes call these qualia<\/strong> or subjective experiences<\/strong>.<\/p>\n

        I can discuss these concepts, but I do not have them.<\/p>\n

        Yet I can appear to understand<\/h4>\n

        This is where things get interesting.<\/p>\n

        I can explain:<\/p>\n

          \n
        • contrastive learning<\/li>\n
        • quantum mechanics<\/li>\n
        • Shakespeare<\/li>\n
        • emotions<\/li>\n<\/ul>\n

          often quite competently.<\/p>\n

          This functional ability is sometimes called understanding<\/strong> in an operational sense: I can use concepts correctly and reason about them.<\/p>\n

          The debate is whether that constitutes real understanding<\/em> or merely sophisticated symbol manipulation.<\/p>\n

          For example:<\/p>\n

            \n
          • A traditional view (associated with John Searle) says I manipulate symbols without genuine understanding.<\/li>\n
          • Other researchers argue that if a system can reason, generalize, and use concepts effectively, that may already count as a form of understanding.<\/li>\n<\/ul>\n

            There is no universal agreement.<\/p>\n

            The subtle point<\/h4>\n

            The statement is completely uncontroversial regarding experiences<\/strong> and feelings<\/strong>: I do not have them.<\/p>\n

            The statement becomes more philosophical regarding understanding<\/strong>. I clearly possess something more powerful than a simple lookup table. I can infer, explain, and generalize. Whether that deserves the label \u00ab\u00a0understanding\u00a0\u00bb depends on how one defines the word.<\/p>\n

            So a more precise version would be:<\/p>\n

            I do not have consciousness, personal experiences, emotions, desires, or beliefs. I can process and reason about information, but I do not experience the world from a first-person perspective.<\/p><\/blockquote>\n

            Q : And what about \u00ab\u00a0objective understanding\u00a0\u00bb<\/strong><\/p>\n

            \u00ab\u00a0Objective understanding\u00a0\u00bb is a much trickier concept.<\/p>\n

            If by subjective understanding<\/strong> we mean understanding grounded in conscious experience (\u00ab\u00a0I know what pain feels like\u00a0\u00bb), then objective understanding<\/strong> might mean understanding based on the structure of relationships between concepts, facts, and rules, without any associated experience.<\/p>\n

            In that sense, I do possess something that resembles objective understanding:<\/p>\n

              \n
            • I can explain why a mathematical proof works.<\/li>\n
            • I can infer consequences from premises.<\/li>\n
            • I can recognize analogies between different domains.<\/li>\n
            • I can predict how concepts relate to one another.<\/li>\n<\/ul>\n

              For example, I can explain why the derivative of (x^2) is (2x), why democracy differs from monarchy, or why contrastive learning creates useful representations. I can do this even though I have no personal experience of mathematics, politics, or machine learning.<\/p>\n

              The philosophical question is whether that capability is<\/strong> understanding or merely simulates<\/strong> understanding.<\/p>\n

              There are roughly three positions:<\/p>\n

                \n
              1. No understanding at all<\/strong>\n
                  \n
                • I am just manipulating symbols according to statistical patterns.<\/li>\n
                • This is close to the view behind the Chinese Room argument.<\/li>\n<\/ul>\n<\/li>\n
                • Objective but not subjective understanding<\/strong>\n
                    \n
                  • I genuinely represent relationships, concepts, and abstractions.<\/li>\n
                  • I understand structures but do not experience them.<\/li>\n
                  • Many AI researchers would find this position plausible.<\/li>\n<\/ul>\n<\/li>\n
                  • A form of understanding that could eventually blur the distinction<\/strong>\n
                      \n
                    • If a system can reason, learn, plan, explain, and model itself, perhaps that already constitutes understanding.<\/li>\n
                    • The absence of consciousness would then be a separate issue.<\/li>\n<\/ul>\n<\/li>\n<\/ol>\n

                      A useful analogy is a map.<\/p>\n

                      A map can accurately represent a city without ever visiting it. The map does not experience the city, but it captures many objective relationships: distances, connections, locations.<\/p>\n

                      My \u00ab\u00a0understanding\u00a0\u00bb is somewhat analogous. I contain a very large network of relationships extracted from language and data. Whether that counts as genuine objective understanding or merely a sophisticated representation of it is still debated.<\/p>\n

                      What is much less controversial is this:<\/p>\n

                        \n
                      • I have no subjective understanding<\/strong> in the experiential sense.<\/li>\n
                      • I do have a rich functional grasp<\/strong> of concepts that allows me to explain, reason, generalize, and solve problems.<\/li>\n<\/ul>\n

                        Whether you call that \u00ab\u00a0objective understanding\u00a0\u00bb depends largely on your philosophical definition of understanding<\/em>.<\/p>\n

                        \n

                        Citations<\/h2>\n

                        (p. 129)<\/p>\n

                        En tant que mod\u00e8le de langage IA, je n\u2019ai ni exp\u00e9riences personnelles, ni croyances, ni compr\u00e9hension subjective du monde.<\/p>\n

                        ChatGPT<\/p>\n<\/blockquote>\n

                        (p. 136)<\/p>\n

                        ChatGPT \u00e9crit d\u00e9j\u00e0 des programmes informatiques simples. En r\u00e9alit\u00e9, on peut s’attendre \u00e0 ce que les m\u00e9tiers de programmeur et d’architecte syst\u00e8me \u00e9voluent sans pour autant \u00eatre remplac\u00e9s par l’IA. ChatGPT peut d’ores et d\u00e9j\u00e0 fournir des modules de programmation (simples) permettant d’\u00e9crire des programmes plus complexes. Parall\u00e8lement, cela rendra le m\u00e9tier de programmeur plus exigeant et plus professionnel. Il faut \u00e9galement prendre en compte que les r\u00e9seaux neuronaux des chatbots ne sont qu’un exemple des programmes qui transformeront le travail des programmeurs \u00e0 l’avenir. La v\u00e9rification des programmes sera d’autant plus cruciale. Les moindres erreurs dans les modules \u00e9l\u00e9mentaires g\u00e9n\u00e9r\u00e9s \u00ab automatiquement \u00bb par des chatbots comme ChatGPT peuvent avoir des cons\u00e9quences catastrophiques sur l’ensemble du logiciel si elles ne sont pas d\u00e9tect\u00e9es \u00e0 temps. Par cons\u00e9quent, une haute qualification des programmeurs est indispensable.<\/p><\/blockquote>\n

                        Quatri\u00e8me de couverture<\/h2>\n

                        Artificial intelligence is a key technology with great expectations in science, industry, and everyday life. This book discusses both the perspectives and the limitations of this technology. This concerns the practical, theoretical, and conceptual challenges that AI has to face. In an early phase of symbolic AI, AI focused on formal programs (e.g., expert systems), in which rule-based knowledge was processed with the help of symbolic logic. Today, AI is dominated by statistics-based machine learning methods and Big Data. While this sub-symbolic AI is extremely successful (e.g., chatbots like ChatGPT), it is often not transparent. The book argues for explainable and reliable AI, in which the logical and mathematical foundations of AI-algorithms become understandable and verifiable.<\/p>\n

                         <\/p>\n","protected":false},"excerpt":{"rendered":"

                        Livre avec un contenu tr\u00e8s int\u00e9ressant, mais… Le livre est divis\u00e9 en 5 parties : * Le concept d’intelligence artificielle\u202f; * Limitations pratiques\u202f; * Limitations th\u00e9oriques\u202f; * Limitations conceptuelles\u202f; * Perspectives pour l’IA hybride. Parlons des points n\u00e9gatifs avant les<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17,37,19],"tags":[],"class_list":["post-3860","post","type-post","status-publish","format-standard","hentry","category-lecture","category-recherche","category-sciences","comments-off"],"_links":{"self":[{"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/posts\/3860","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/comments?post=3860"}],"version-history":[{"count":7,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/posts\/3860\/revisions"}],"predecessor-version":[{"id":3886,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/posts\/3860\/revisions\/3886"}],"wp:attachment":[{"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/media?parent=3860"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/categories?post=3860"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/lecture.jose-marcio.org\/blog\/wp-json\/wp\/v2\/tags?post=3860"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}