Developing a Taiwanese Legal Expert Response System Using Generative Artificial Intelligence and Retrieval-Augmented Generation

Yeu-Pong Lai , Chia-Ling Chen, Chen-Yu Li

Recently, the Large Language Models (LLMs) have emerged as “knowledge consultants” in many fields, such as Natural Language Processing and Generation, Customer Service and Chatbots, Legal Field, Healthcare, Education Assistance, Financial Services, Content Creation, and Recommendation Systems and Search Engines. In the Legal Field, the LLMs can be applied in legal document analysis, case retrieve-al, contract review, and legal consultation to help lawyers and judges improve work efficiency and accuracy. This study aims to develop a legal expert question-answering system tailored for Taiwan's judicial sector by leveraging generative artificial intelligence and retrieval-augmented generation (RAG) techniques. By integrating a local LLM service with a vector database of judicial documents in certain domains, the system seeks to enhance the accuracy and efficiency of legal case investigation and consultation. With the advancement of Transformer architectures and LLMs such as BERT, LLMs have demonstrated outstanding language understanding and generation capabilities. However, they still face challenges like hallucinations and outdated knowledge. This research employs RAG technology to combine retrieval and generation, effectively reducing hallucinations and supplementing the latest information. The study uses 30 Chinese judicial rulings and indictments as data, applying Google Gemini and Meta LLaMA language models for text segmentation and vectorization. Experiments were conducted with various chunk sizes ranging from 64 to 1024 characters and different overlap ratios. The processed token texts were stored as 768-dimensional vectors in a Pinecone database. Additionally, two n8n workflows were designed to implement the two main processes of token vectorizing and generative question answering. During the experiments, NotebookLM is introduced to generate 100 legal questions covering specific case features, legal concepts, and general inquiries. The results were categorized into emergent intelligence, true positives, false positives, and false negatives. The findings indicate that a chunk size of 128 characters without overlap yielded the best performance, effectively preserving context and providing longer, more accurate answers. Conversely, large chunk sizes or too much overlap caused information overload for the model, reducing query efficiency and accuracy. The study further compared token usage across different groups and found that the number of prompt tokens increased as the chunk size decreased, while the response length tended to stabilize under reasonable overlap settings. This suggests that the segmentation strategy has a direct impact on the performance of the RAG system. To ensure data security and prevent sensitive information leakage, the system adopts a local deployment strategy, combined with the highly flexible n8n workflow, effectively integrating the local language model with the vector database. This research not only validates the feasibility of applying RAG technology in the Chinese legal professional domain but also emphasizes the critical impact of text segmentation methods on answer quality. It provides important references for future expansion of judicial databases and enhancement of language models, aiming to promote the digital transformation of the intelligent judiciary and police investigation.