Technical Research and Architecture Design for a Conversational Search System PoC Project, Aug. 2025 ~ Sep. 2025
1. Background
At the request of an AI vendor serving a major manufacturing enterprise as its client, I participated as a researcher and advisor in a Proof of Concept (PoC) project for a “conversational, high-precision search system to propose countermeasures for machine failures and malfunctions.” The objective of this project is to apply various RAG (Retrieval-Augmented Generation) technologies, including GraphRAG, to verify improvements in conversational response accuracy.
2. Implemented Content
Based on the project requirements, the following technical investigation, evaluation, and system design were conducted.
2.1. Systematic Evaluation of RAG Approaches
First, evaluation criteria for RAG technologies were established based on the project requirements.
- Evaluation Criteria:
- Feasibility (within the PoC period, possibility of construction with existing technology)
- Suitability for Domain and Data Characteristics
- Scalability and Maintainability
- Implementation Difficulty
- Robustness (Noise Tolerance)
- Real-time Performance
Based on these criteria, eight state-of-the-art RAG approaches, including ToG (Think-on-Graph), GraphRAG, LightRAG, and RAS (Retrieval-And-Structuring), were analyzed for their respective advantages and disadvantages using primary sources such as academic papers, followed by an objective comparative evaluation with scoring.
2.2. System Architecture Design
An architecture for the system to be built in the PoC was designed based on the evaluation results.
- Basic Configuration:
- Knowledge-Base Layer: An independent API server layer to build and manage a knowledge graph from a document corpus. The adoption of LightRAG was proposed.
- LLM Orchestration Layer: An application layer to manage the conversational flow with users and calls to external tools, assuming the use of frameworks like LangChain/LlamaIndex.
This two-layer separated structure was intended to simplify development and tuning, and to improve scalability and maintainability. Furthermore, a specific three-step pipeline for automatically constructing a knowledge graph from unstructured text was defined (Entity/Relationship Extraction, Structuring, and Deduplication/Optimization).
2.3. PoC Development Plan Formulation
Based on the designed architecture, a concrete milestone plan was formulated to build a Minimum Viable Product (MVP) within two months.
2.4. Response to Technical Inquiries
For technical questions from project stakeholders regarding specific technologies like GNN-RAG, detailed explanatory materials were created and provided, citing research papers to explain their principles, advantages, and use cases.
