Prompt. Plead. Proceed. But Who Answers When Justice Is Questioned

By,
Adv Sherry Samuel Oommen: This article is authored by Adv Sherry Samuel Oommen. He special
ises in the Constitution, tax and corporate laws and has also cleared the final exams of the In
stitute of Chartered Accountants of India, the Institute of Cost Accountants of India and the
Institute of Company Secretaries of India. He has also completed his Masters’s Degree in Com
merce, apart from obtaining a Post Graduate Diploma in Business and Corporate Laws from Sym
biosis, Pune. The views expressly are personal and should not be construed as a legal opinion.
sherryoommen@nashcp.com

Earlier, a legal problem usually began with a lawyer. Often it begins today, with a prompt. “My landlord hasn’t returned my deposit. What remedies do I have?” “Should I file a 498A case?”” “Write a good legal notice. “Give me Supreme Court decisions that support my case.” In the blink of an eye there is an answer. It is neat, legal sounding, trustworthy and full of sections, case names and legalisms. Looks very convincing enough. And somehow along the way we seem to have acquired a new kind of personalized lawyer, the one sitting inside almost every Indian’s phone.

There is something genuinely useful in this. Law has consistently posed a language challenge. Statutes, judgements and procedures aren’t exactly written for the ordinary man.  Artificial Intelligence (“AI”) has made it easier for people to understand a notice, read a provision or even initiate the right questions. You don’t intend to discount that part. But there is a very important distinction that exists between understanding the law and understanding a legal problem. A machine can often help with the first. The second requires much more than an answer in seconds.

The issue starts when a well-written answer becomes mixed up with a correct answer. Those who have actually practiced law know how often the answer to a legal question begins with, “It depends. Depends on the facts. Depends on document.

It is an inquiry of what happened prior to the litigation reached court. It could be a limitation, evidence, standard procedure, jurisdiction or a judgement that appears irrelevant until you have read it properly. AI has an innate desire to just tidy all of the uncertainty away. Give it a complicated argument and it will effortlessly give you five headings and an outcome. The problem is the law is rarely so simple.

The Indian system of justice, in particular, cannot be considered as a mathematical exercise. It really matters not if the person making the inquiry is sympathetic or the circumstances are unusual.  If a mathematical problem is done precisely, the answer is uniform.  The courts deal with people, and people don’t always come in neat compact packages. Two cases may pertain to the same provision but nevertheless come to different conclusions because the proof presented is different, the conduct of the parties is distinct or the procedure implemented in one case was fundamentally unfair. A judge is not merely asking what Section X actually says. The court wants to know what happened, has it been proved, had it been heard on both sides, does the previous precedent actually apply, what consequence should the law confer in those particular circumstances? That is where principles of natural justice, fairness, equality and judicial discretion come into play. You don’t exclusively include these variables into a prompt.  The concern becomes much more serious when the system starts delivering authorities which do not exist. It’s not a hypothetical possibility anymore.

The Supreme Court, in Pooja Ramesh Singh v. Jammu and Kashmir Bank Ltd. AIR 2026 SC 3247 examined the dilemma of AI-generated and hallucinated legal authorities engaging in the adjudicatory process. Automated language models are not legitimately accountable for creating legal precedents in court filings; the legal professional bears the ultimate responsibility. Therefore, AI support is no excuse for misrepresenting legal authorities to the courts. Pre-NCLT authorities cited and relied upon were non-existent, wrongly cited or not consistent with actual judgements. The decisions of the NCLT and NCLAT were ultimately quashed. What makes the case particularly important is not just that an AI system got a parameter wrong. It is that something generated by a machine was presented as law and made its way into a judicial decision-making process. The Supreme Court’s solution was not toss out to technology in toto.  It was to stress verification and personal accountability. That seems an appropriate line to draw: use the equipment, but refrain from trying to outsource the judgment.

Which ultimately leads to a probing question. Who pays for an AI mistake? The machine, definitely not. It does not look to the Bench when the judge asks the origin of a particular judgment. This doesn’t explain why a case that has no existence appeared in writing. It has no professional consequences. The user does. Apologies but “ChatGPT told me so” is not a professional defense pertaining to placing wrong law before a court. Technology can tell you how the error occurred. It doesn’t automatically carry over the obligation for the error.

There is another issue that may become more apparent as AI becomes part of everyday litigation. With AI it is incredibly easy to produce legal materials. It now takes less time for a person to draft a long petition, list dozens of grounds, quote provisions and generate case citations than it would have taken previously to start researching the issue. But the court still has to read the stuff. The thirty pages were produced in thirty seconds, so a judge can’t verify thirty pages in thirty seconds. Somebody has to separate the wheat from the chaff, vet the authorities and find out what the real dispute is. If this is prevalent, there is a substantial chance that some of the duties, that traditionally fell on lawyers in preparation and verification of legal material, will shift toward the judiciary. A court should settle disputes. It should not have to waste time determining whether the instances cited in a pleading even exist.

This is also why the phrase “AI lawyer” is somewhat misleading. A lawyer is not just someone who knows the rules and can write a convincing paragraph. The lawyer reads the documents, examines the facts, knows the procedural history, evaluates the evidence, anticipates the other side and, most significantly, takes responsibility for the advice given. Sometimes the most beneficial thing a lawyer can do for a client is not to say “Here are ten grounds to file a case”, but “Don’t file this case; your evidence does not support it.” Sometimes the real issue is not the law at all. It is a missing document, an unreliable witness, limitation or one inconvenient fact that changes everything.

You can’t measure that kind of judgment by how competent the answer looks on a screen. Then there’s the data we give out.

When a person tells an AI to “analyse my case” they are rarely presenting a hypothetical problem. They can upload divorce petition, Aadhaar card, medical records, WhatsApp conversations, bank statements, photographs, employment records, children’s details or an entire bundle of correspondence. That may amount with years of an individual’s private life in a family dispute. In a criminal matter it may comprise statements, captures and medical material. In an employment dispute, it could include salary details, internal communications as well as individual records.

This is where the Digital Personal Data Protection Act, 2023 (“the Act”) comes into play for the discussion. The Act lays down a framework for processing electronic personal data and imposes obligations on such processing. The point of it in relation to AI is not just to say “don’t put your personal information in AI”. The issue in question is more elementary. As more people use AI for things they would traditionally talk to a lawyer about, they are also incorporating more and more sensitive information into electronic systems. Hence the legal question is not exhausted by the answer the machine gives. And behind that there is another question: what happens to that information that was used to produce that answer.

Under the Act, an entity that collects and processes such information ordinarily functions as a “Data Fiduciary” and is required to process personal data only for a lawful purpose for which the individual has given free, specific and informed consent, and to collect no more data than is necessary for that purpose. In practice, a user who uploads an entire case file, medical record or set of WhatsApp chats to a chatbot rarely receives any meaningful notice of what is collected, how long it is retained, whether it feeds into training the underlying model, or with whom it may be shared. A generic terms-of-service click is a poor substitute for the kind of informed, purpose-specific consent the Act contemplates.

The average person who asks an AI to be their attorney probably isn’t going to read a privacy policy first. They are worried about the divorce, the familial property dispute, the employment termination or the police complaint in front of them. They want an immediate answer. That is precisely why the question of privacy deserves utmost attention to detail. We have spent years and ages  telling people that lawyers have professional obligations when handling confidential information. We now need to ask whether the same person understands what happens when that information is moved from the lawyer’s office to an AI platform. The question, therefore, is no longer only, “Is the answer legally correct?” It is also, “What happened to the information I gave you in order to get that answer?”

There is also no equivalent of legal professional privilege once information leaves a lawyer’s hands and enters an AI platform. Communications with a lawyer are protected as confidential; the same information typed into a chatbot enjoys no such protection, and may be stored on servers located outside India, processed by entities not readily amenable to Indian jurisdiction, and potentially retained even after the conversation ends. Matrimonial disputes, criminal complaints, medical records and children’s details are precisely the kind of information where unauthorised processing, a breach or a cross-border transfer can cause real and lasting harm, yet the Act does not carve out a distinct, higher-protection category for such information in the way earlier data protection frameworks attempted to. Until the position is tested, users would be well advised to redact identifying details before relying on AI tools for anything resembling legal advice.

Here the discussion about Artificial Intelligence goes beyond legal research. It becomes a conversation about privacy, autonomy and the control that an individual has over the most intimate information about their own life.

That’s not to say AI has no place in the justice system. Not in the least. It can be a great help if used with care. It can help make legal material more accessible, help with translation, help sort through large volumes of material and allow people to ask questions they might otherwise be too intimidated to ask. For the person who who’s never been in court before, that can make a system that’s otherwise terrifying just a little bit less out of reach. And that’s a real value.

But the line should be drawn between the legal literacy of AI-assisted and the legal judgment of AI-substituted. The distinction is important because the basis for our system of justice is human nature itself. The concept of natural justice is not a mere formality. A fair hearing cannot be simply a formality. Equal treatment cannot always be the same treatment. Sometimes the court must consider the context of the dispute before it determines the legal requirements. Sometimes the court will have to consider whether there was really a hearing, whether the process was just, whether a specific remedy should be provided, or whether the application of a certain rule would lead to an unfair outcome. Thus, perhaps the question should not be whether AI will take the place of lawyers but rather whether we can figure out how to use AI without becoming convinced that AI has made us lawyers.

With the ability of all of us to produce a legal notice, a petition, and citations to twenty cases, the skill of producing legal language loses its importance. The skill becomes crucial when you know which judgement to apply, what fact changes your case, what argument to make, when the law cannot rescue the evidence and what argument to drop, when the evidence is stronger than the law. And sometimes, simply knowing when to say, “I don’t know. Let me check.” That may turn out to be one of the most important professional skills of the AI era.

As the Indian judiciary system was not designed as a machine which will receive facts and deliver judgments. It is a human institution, handling human conflicts, and striving hard to deliver justice as best as possible. Technology will definitely help the Indian judicial system. Technology will make the lawyer’s job easier and information available more easily. However, speed cannot be the standard of justice, and an answer cannot be substituted for a judgment.

A citation does not equal an argument; an argument does not equal evidence; evidence does not mean truth; and even a legally correct proposition may not be the end of the matter.

The law is not all about reaching an answer but about deciding what the law requires when human beings are involved in a case. Therefore, perhaps the true danger is not that everyone now has a lawyer in their phone.

It is rather that everyone will soon think that they have one.

A prompt can spark a debate.

An algorithm can formulate a lawsuit.

A database can uncover a verdict.

But when the doors of the courtroom close and two human beings stand on opposite sides, none of those things can carry the responsibility of deciding what is fair.

AI can process the law.

Justice still has to be judged.

Prompt. Plead. Proceed.

But never forget who has to answer when the question finally becomes: Was justice done?

Pic courtesy: google/ images are subject to copyright