Shourya Shekhar and Vibhuti Shyam
Summary: This article contends that the Supreme Court’s Draft Regulations for Use of Artificial Intelligence in Courts, 2026 is a principled but architecturally incomplete first step. While the draft correctly prohibits the most constitutionally dangerous uses of AI, three structural gaps mean its core commitments lack the architecture to meet Articles 14 and 21 of the Constitution: an undefined “high-risk” category, an audit mechanism that cannot inspect the systems it oversees, and a remedy limited to prohibited uses.
The world is evolving at an unprecedented pace and it is imperative that the Judiciary of a country does not lag behind and thus, leave a lacuna in the country’s governance framework. In this regard, the Supreme Court of India recently issued Draft Regulations for use of Artificial Intelligence (“AI”) in Courts, 2026, which clearly states that while AI may aid the judicial process, it cannot replace human decision-making. The regulations are in line with a wider global trend to integrate Artificial Intelligence into judicial and legal processes, while protecting the fundamental institutional principles that form the basis of the administration of justice. The discussion around how AI should be used in deciding cases today began back in May of 2023, when the US District Court for the Southern District of New York sanctioned a lawyer for submitting documents that were based on fictitious court cases referenced by an AI-based chatbot, in the case of Mata v. Avianca.
India has not faced such a situation in its own courts, and the Draft Regulations show an intent to keep it that way. The Draft Regulations represent a genuine attempt to regulate an industry already operating and growing in India’s courts. This article argues that the Draft Regulations are an appropriate beginning but structurally incomplete on implementation. They also prohibit some of the most constitutionally dangerous uses of AI. Examples of this are: “Risk Scoring to determine Bail and Sentencing,” and “Completing Automated Proceedings.” This regulatory framework will position India as one of the more forward-looking jurisdictions in the world on AI in law. However, the three large deficits will create challenges in adhering to these Draft Regulations.
The draft does not define what is meant by a “high-risk” AI application. It has an audit mechanism structurally incapable of auditing the very systems it is meant to govern. And it limits litigant remedies to a subset of AI-related harms that is too narrow to satisfy what Articles 14 and 21 of the Constitution require. If not addressed, these structural gaps will leave the draft’s most important commitments without the institutional architecture to give them practical effect.
What The Draft Gets Right
The draft’s absolute prohibition on Risk Scoring is one of its strongest provisions. Draft Regulation 20(d) outright prohibits use of AI tools that generate risk scores for bail, recidivism, flight risk or sentencing recommendations. In Loomis v. Wisconsin, defendant Eric Loomis was sentenced in part based on a score generated by Correctional Offender Management Profiling for Alternative Sanctions (“COMPAS”), a proprietary algorithmic tool designed to assess the likelihood of an individual reoffending. However, the defendant was refused access to the underlying methodology of the algorithm, and was effectively sentenced by reference to a number whose generation he was unable to interrogate, verify or meaningfully challenge. ProPublica’s 2016 investigative report, “Machine Bias” subsequently documented significant racial disparities in how COMPAS scores were assigned across demographic groups.
The constitutional problem as viewed through the Indian prism is much larger than merely evidentiary problems. The right to a meaningful hearing is included in the right against arbitrary deprivation of liberty guaranteed under Article 21 of the Constitution, a position consistently taken by the Supreme Court. A bail decision made using an algorithmic score (even if it is used only as a recommendation) but where the accused cannot see or challenge the score or the factors affecting the score, would be contrary to Due Process and a fair trial. If any part of the basis for making an assessment about a person’s liberty from a private algorithm is “hidden,” then the decision is not “reasoned.”
Regulation 4 and 20(b) and (c) state that no AI system can provide a judicial determination or sentence to an accused. Only the judge has the authority to adjudicate; this is not merely technical reading of the regulations.
A number of jurisdictions, citing efficiency gains, have let AI fulfil judge-like roles at hearings and in concluding cases. The Indian draft refuses to take that path simply because the Indian constitutional framework requires human deliberation at the moment of judgment.
Behavioural researchers have documented extensively a phenomenon called ‘automation bias’. This is about the tendency of human decision-makers to defer to algorithmic outputs, particularly when these outputs are numerically precise and appear authoritative. In any meaningful sense, a judicial officer, formally the decision-maker, who is presented with an AI-generated recommendation that he or she lacks the technical means to meaningfully evaluate, is not exercising independent judgment. The substance of human primacy is not preserved, only the form. Regulations 4 and 20(b)-(c) address this head-on, requiring real human primacy. The principle itself is correctly conceived. It is a separate as well as a troubling question if the rest of the statutory regulations have been suitably operationalised.
Regulation 43 has a provision that requires any advocate using an AI application programme (software) in preparing a pleading to inform the court of its use. The advocate bears full legal responsibility for the pleading’s accuracy, and cannot raise use of an AI tool as a defence to liability for errors it introduces.
The regulation’s intention is to address the lessons learned from India’s regulatory response to Mata v. Avianca. The logic is clear; if a practitioner chooses a tool (AI) that generates fictitious or non-authoritative sources with a richness of detail/creativity that is indistinguishable from a legal researcher, they assume the duty to ensure that the output is accurate before placing it before the court. An advocate has a duty to assure candour towards the court under the Advocates Act, 1961, and the Rules of the Bar Council of India, and cannot blame the AI application for an error of judgment.
These three rules form what could be called a constitutionally-grounded minimum. They comprise a guarantee of a fair trial per Article 21; they meet the equality principles required by all state authorised action under Article 14; and they indicate a clear understanding of the problems of hallucinated/opaque/automation bias that can be introduced into legal systems by AI. However, this structure has significant flaws that could undermine it.
Persistent Gaps
The prohibitions set out here (Regulation 20), the human-priority requirement (Regulation 4), and the disclosure requirement (Regulation 43) provide a skeleton that could support solid judicial governance of AI. The draft then attempts to build on that skeleton. There are provisions enabling an apex body to set national standards, mandatory annual audits of all deployed AI systems, proportionality and transparency requirements for higher risk applications to ensure accountability, and a mechanism for seeking recompense by litigants who have been adversely affected by judicial AI. The issues arise from the operative provisions proposed to achieve these lofty goals.
The Undefined Category Of ‘High-Risk’ Applications
The operative provisions of the draft on transparency and proportionality are Regulations 7 and 12. Regulation 7 requires AI systems used in high-risk applications to be more transparent and explainable. Regulation 12 requires that AI deployment be proportionate to the risks and that tools affecting personal liberty are subject to commensurately heightened security. Though the provisions are sensible in principle, it is the structure that is fatal to their enforceability. Neither Regulation 7 nor Regulation 12 nor any other provision in the draft provides a definition of what constitutes an AI application being “high-risk”. The phrase recurs throughout the regulations as if its meaning were self-evident. There is no list of high-risk applications, no criteria by which a given tool could be assessed, no designated body empowered to make classification decisions in contested cases, and no procedure by which a classification might be challenged or appealed. The proportionality principle contained in Regulation 12 is therefore almost impossible to enforce.
A Court Administrator introducing a new AI-assisted transcription system cannot tell from the regulations alone whether it falls within the heightened requirements of Regulations 7 and 12. The system would require detailed explainability documentation and prescribed standards of transparency within the high-risk category. Those requirements do not apply outside it. The draft provides no basis for determining the question, leaving the most consequential regulatory decisions to ad hoc judgment.
This definitional problem was confronted by the European Union in the legislative process that produced the European Union’s Artificial Intelligence Act (the “EU AI Act”), Regulation (EU) 2024/1689. The structural solution consisted of an exhaustive list of high-risk AI systems tailored for each sector, along with a detailed description of how AI systems can administer justice. India cannot simply copy this structure, given how differently its courts operate and how much internal variation exists within the country. However, the need for a very systematic and structured classification of AI systems along sector lines is applicable and mandated.
A three-tier classification system can serve as both a model and a ready-made foundation for India’s regulatory framework. Not as a conceptual proposal, but as draft language that could be inserted directly into the Regulations. A sample provision might read as follows:
“AI systems used in courts will be classified into three tiers: (1) Administrative-Low includes systems used for scheduling and generating cause lists and notices; (2) Judicial-Medium includes systems used to assist with legal research, translating and transcribing; and (3) Liberty-Affecting-High includes any AI system with a direct or indirect output that will affect decision-making concerning bail, custody, or sentencing and any other order regarding a person’s legal rights. Tier 3 Systems shall be required to comply with the increased levels of transparency and explainability outlined in Regulation 7.”
Including this classification in the Regulations; rather than leaving high-risk undefined across Regulations 7 and 12 would give courts, vendors and litigants a shared vocabulary to apply in advance, rather than working it out informally after a dispute arises. This mirrors the EU AI Act, whose Annex III defines high-risk classes with sectoral specificity, giving it clear boundaries and an institutional mechanism with authority over borderline cases; safeguards that India’s draft currently lacks.
The Draft That Is Blind Inside
The Draft Regulations include Regulation 38, a highly ambitious regulation. Sub-regulation (1) requires that all AI systems used in court proceedings undergo periodic technical, legal, and ethical audits no more than a year apart. This requirement makes clear that approval at deployment does not guarantee an AI system to operate reliably, lawfully or fairly.
The main concern regarding Regulation 38 is contained in sub-regulation (2). This contains language that requires all audits to be done in-house and states that source code, algorithms, datasets or other architectural information may not be shared with any other parties for any reason. This restriction is necessary given the sensitivity of judicial data, the proprietary nature of AI system design and the potential consequences of disclosure. But the provision is worded absolutely. By blocking all external access, it essentially limits the upper bound of what a meaningful audit can achieve, regardless of the approach used.
Regulation 41 will require the Supreme Court’s own AI-assisted legal research tool, the Supreme Court Portal for Assistance in Courts Efficiency (“SUPACE”), to be reviewed for compliance within one year of commencement. Most AI systems likely to be deployed in Indian courts are commercial or semi-proprietary in design. In-house auditors at the AI Secretariat can look at the inputs and outputs, write down the error rates and run performance tests for such systems. What they don’t see is the underlying architecture that explains the system’s outputs. An output audit can verify that a translation tool is producing errors at a particular rate. It cannot know if those errors are randomly distributed, or systematically concentrated against certain languages, communities or socioeconomic categories. The first is an operational one. The second is constitutional.
This is where the gap becomes constitutionally salient. Article 14 requires state action, including state action undertaken with the help of state deployed technology, to be non-arbitrary. An AI based tool that unfairly considers one party’s claim against another unwittingly violates the equal treatment of parties as guaranteed by the Constitution. Article 21 guarantees fair process; a litigant who cannot examine or challenge bias in an AI system has no way to know whether the system has compromised their right to a fair hearing. The issue is not whether AI can provide value to the litigation process; instead, it is how the use of AI to assist litigation in an unchallengeable manner violates due process.
The AI Secretariat already has a contractual entitlement to “audit and inspect” AI systems and data from vendors in accordance with Regulation 46(4)(e). Regulation 38(2) renders that right hollow; auditors cannot meaningfully exercise it without the technical access needed to evaluate the underlying data.
The EU AI Act conformity assessment system as mandated in Articles 43 and 44 provides for an evaluated structure for the pre-implementation and continual verification of high-risk AI systems and serves as a structural model of what is necessary rather than how to implement a similar regime in India. Under that framework, independent accredited bodies bound by strict confidentiality obligations are given standard access to training and validation datasets. The problem of auditing proprietary AI without full public disclosure has been solved elsewhere with regulated independence rather than absolute prohibition.
A model amendment to Regulation 38(2) might read:
“For AI systems categorised as tier (iii), accredited external auditors working under judicially supervised non-disclosure agreements may have access to such architectural information as is strictly necessary for the purposes of audit, with the findings reported only to the AI Secretariat under seal.”
Such a provision would not open up judicial systems or vendor architecture to public scrutiny. It would provide the AI Secretariat with the institutional standing to move beyond output observation, to determine whether a system like SUPACE produces research outputs that are structurally skewed against certain legal positions, languages or litigant categories, and to require the vendor to address the cause rather than the symptom. Courts, vendors, and litigants would each have a shared assurance that audit means more than a performance log. The answer is independence under control, not disclosure without limit. It is the least institutional condition for the word “technical” in Regulation 38(1) to mean what it says.
The Cure That Misses The Point
Regulation 52 is a productive acknowledgement that AI in the courtroom could have significant negative impacts on real world people. Sub-regulation (1) establishes a way for individuals to file complaints with the court where an AI system was used and, after a hearing, allows the court to take appropriate action. Sub-regulation (2) further directs High Courts to establish easy-to-follow processes for filing complaints that take into account the inability of some litigants to read and write legally. This is a real achievement for building an institutional process to recognise that litigants could be victims of AI-based decision making in the courtroom.
The issue is with how the grievance process works. Regulation 52(1) limits the grievance process to people harmed by AI use that violates Regulation 20. This is a defining limitation of the mechanism. Conversely, it does not apply to harm caused by a court’s permitted use of AI, regardless of severity.
Picture a court utilising an AI translation device approved by Regulation 19(c) that mis-translates legal pleadings in a minority language. How can a litigant whose case was litigated based on a dangerous translation have no remedy within the meaning of Regulation 52 in that regulatory framework? The same holds for an AI transcription device deployed under Regulation 19(b) that consistently mishears certain accents or dialects, permanently distorting the court record. The record stands; the error is lawful; the harm is real.
These are not exceptions. They are precisely the failures that occur most frequently in the context of AI systems deployed at scale in linguistically diverse populations and India’s courts have perhaps the most linguistically diverse litigant population in the world. That reality is the root of the constitutional problem. Formal equal access to an AI-assisted process does not fulfil the guarantee of equality before the law in Article 14 if the process systematically leads to worse outcomes for speakers of certain languages, members of certain communities or litigants from a certain socioeconomic background. The formal deployment of a permitted tool does not relieve the state of its obligation of substantive equality. This is reinforced by Article 21: a litigant who is unable to obtain correction of a procedurally significant AI error has not had a fair hearing in any meaningful sense, the right to be heard carries within it the right to be heard accurately, and a court record that misrepresents what was said is not a record of a fair proceeding.
The “other remedies” provision of Regulation 53 does not adequately fill this gap. The primary avenue of challenge for a translation or transcription error is writ jurisdiction, and that avenue imposes a cost, financial, temporal, and technical, that most affected litigants cannot bear. It is the semblance of access to justice, not the reality.
According to the UNESCO Guidelines for the Use of AI Systems in Courts and Tribunals, under the principles of accountability and human oversight, the redressal mechanisms should include AI-related harm in judicial processes, whether the deployment is lawful or unlawful. In Brazil, the Supreme Federal Tribunal’s own AI-assisted case management system, Victor AI, has a feedback and correction mechanism that allows parties affected by an AI-generated output to flag errors, precisely because permitted use does not necessarily mean accurate use. The structural conclusion of both frameworks is the same: the boundary of a grievance mechanism should track the boundary of potential harm, not the boundary of prohibited conduct.
The necessary reform is to extend Regulation 52(1) to cover harm caused by any AI system deployed in court processes, whether permitted or not. For example, a model amendment might be:
“Where a party to a proceeding is harmed as a result of the use of any AI system in that proceeding, whether or not such use amounts to a prohibited use under Regulation 20, that party may apply to the court for appropriate relief.”
This should be complemented by a designated AI grievance officer within each High Court’s AI Secretariat, with the power to receive, record and escalate complaints about repeated AI failures, developing an institutional memory that links individual errors to systemic patterns before those patterns inflict further damage.
Three Comparative Lessons For India
The gaps identified in India’s initial attempt to govern judicial AI are not unique. Other jurisdictions have encountered, and to some extent resolved them. The comparative frameworks below offer a foundation and evidence that the institutional structures proposed here are realistic.
The first lesson learned relates to proportionality. The European Union’s AI Act does not classify all artificial intelligence systems alike. As stated in Annex III of the Act, AI systems will be categorised by usage sector, and because judicial decision-making will be listed, tools used will be subject to the highest levels of requirement relating to their transparency and human oversight before their deployment in court. This is logical; the more consequential the application of the tool, the greater the demands it must meet, compared with tools that do not carry the same impact. The draft statute of India endorses the concept within Regulations 7 and 12; however, Regulation 12 has been shown not to implement the proportionality principle established by the AI Act so that the principle of proportionality will become a rule and no longer simply an expectation.
Lesson two centres on the requirement for human oversight to ensure the quality of AI-enabled judicial decision-making. The UNESCO Guidelines for the Use of AI Systems in Courts and Tribunals establish the requirement that the human respondent to an AI-generated decision has a thorough understanding of the AI system and is capable of questioning, and correcting any erroneous outputs produced by the AI system. The essential difference between what the Guidelines describe as ‘meaningful control’ and ‘nominal supervision’ corresponds directly to the automation bias issues raised by Regulations 4 and 20(b)-(c) but not addressed by those regulations. If a judicial officer cannot question an output generated by an AI research tool, that judicial officer does not provide meaningful oversight of the AI-generated output. Capacity-building provisions contained in Regulation 49 offer a nod towards addressing this issue but true human oversight requires institutional competency rather than simply an institutional presence.
The third lesson for judicial AI governance is trust via transparency. One example of this can be found in the Victor Artificial Intelligence application created by the Supreme Federal Tribunal of Brazil for the automated triage and management of cases. The Victor system publishes reports on transparency, has a limited operating scope and will provide a means of correcting errors made by the AI-generated output that impacts parties. Thus, public trust in the use of Artificial Intelligence in judicial systems must be earned through holding the parties who create the system accountable; thus, the design principle is that validation of accountability is the basis for establishing public trust in AI.
India’s Regulation 45 provides for annual reporting on transparency. This regulation is based on the same premise. However, Regulation 38 exposes the audit void in the transparency report process; thus, it demonstrates how a good intention of reporting on transparency can become simply an exercise of reporting rather than establishing accountability when the report is not independently verified and there is no available correction process for the affected parties.
The Victor framework illustrates what that difference looks like in practice and points to the need for proportionality in design, substantial rather than merely formal human oversight, and institutional mechanisms that make accountability verifiable rather than declaratory.
Conclusion
The Draft Supreme Court Regulations are not just a theoretical exercise; they are designed to apply an existing technological use in the courts, and make that use accountable through the Constitution. The prohibition on risk scoring, the requirement for human beings to exercise authority, and the mandatory disclosure obligations are not simply showing goodwill to a passing fad; they have been carefully considered based upon what is required by the Constitution.
However, as this article points out, there is a significant gap between the principles involved and the structure to implement them. A structure that blocks the right behaviour while allowing unaccountable conduct, helps to fix the damage done by prohibited conduct, and only places enhanced protection on a designated category that is never defined, still has to achieve its goal of creating an accountable framework for all actions taken in courts and government. The three gaps identified in this article should not make us question the intent of the draft, they should prompt us to improve on the design of the draft, so that when those three areas are put in place, they work together.
The question before the Supreme Court isn’t whether AI itself should be used in the courts; rather it is whether the institutions that govern AI have been structured such that there is an assurance of fairness when someone appears before a court using AI in 10 years.
Shourya Shekhar and Vibhuti Shyam are both B.A. LL.B. (Hons.) students at National Law University, Jodhpur.
NUALS Law Journal 