AI must be used in legal matters only under human supervision !

Supreme Court observation

New Delhi – The use of fabricated legal precedents generated by Artificial Intelligence (AI) is dangerous. The risk is as grave as the toxic gas leak in the Bhopal gas tragedy. Therefore, AI must be used only under human supervision, the Supreme Court observed while hearing a case. The Bench of Justice P S Narasimha and Justice Alok Aradhe made this observation while setting aside an order of the National Company Law Tribunal (NCLT). The Bench stated that presenting false and non-existent judgments generated by AI as genuine before a court harms the justice delivery system. Therefore, courts should show no leniency in such cases.

What exactly did the Supreme Court say about the use of AI in legal matters?

1. Fabricated legal information generated by AI may appear to be a minor issue at first glance, but it is extremely dangerous. It undermines the judicial process and can also erode public confidence in court judgments.

2. The Court is not opposed to AI technology. The problem does not lie with AI itself but with presenting false information generated by AI as though it were true. Therefore, AI must be used with caution, proper verification, and human supervision.

3. AI may be used as an aid, but the final decision and verification must always be carried out by humans.

4. The Bar Council of India should constitute a committee to frame rules to prevent the submission of fabricated and misleading AI-generated material before courts and to ensure action against those who violate such rules.

What is the case about ?

In one matter, the National Company Law Tribunal (NCLT), which adjudicates disputes relating to companies and insolvency matters, relied on several legal precedents to support its decision. However, many of those cited cases did not actually exist and had been generated using AI. A petition challenging that decision was subsequently filed before the Supreme Court, which made the above observations while hearing the matter.