Joint Research Project on Causal Inference with a Research Institute (LLM), Oct. 2024~Dec. 2025
Background. In a joint project with a research institute, I ran a Proof of Concept (PoC) applying Large Language Models (LLMs) to healthcare-scale big data in order to evaluate causal-inference approaches, with an emphasis on localized and domain-specific deployment under strict data-confidentiality constraints.
A central theme in this field is the prediction of future health conditions, but its practical application faces several hurdles. Data confidentiality is a significant barrier, which is what makes local, domain-specific LLMs valuable here. The project itself also began without a unified framework: evaluation criteria were unclear, ground truth was inconsistent across methods, and there was limited ability to compare heterogeneous methods or to capture temporal effects. On top of this, the budget did not cover the staffing that the project’s scope required.
Implemented content. I served in a dual capacity as both Researcher and Project Manager (PM), and in practice carried the work end to end — research, project management, writing, implementation, and closeout.
Research program and unifying framework. I defined and structured the research program and introduced a unifying framework centered on temporal signals. At an operational level, this meant selecting relevant substructures from a vast, high-dimensional dataset, defining a method-consistent ground truth, and establishing evaluation criteria that align heterogeneous estimators under a common framework. A specific research thrust examined the contribution of item-intrinsic signals — particularly temporality — to causal-effect estimation under intervention.
Reproducible pipeline and prototype. I designed a reproducible pipeline integrating multiple inference methods, and built on it a prototype that performs causal inference between data points and enables comparative analysis across different modeling assumptions.
Project management. I designed the experimental workflows, assigned tasks across the AI engineers on the project, and oversaw their progress through iterative technical reviews.
Delivering under a staffing shortfall. When additional staffing was not available, I adjusted the work cadence to protect both quality and motivation, and kept detailed internal and external records so that the partner research lab could track unresolved needs and coworkers could follow progress.
Results. The PoC delivered a working prototype that enables consistent comparison across causal-inference methods and improves the interpretability of causal effects — despite the limited staffing. Its structurally interpretable results informed both methodological questions in causal inference and domain-specific medical considerations. The project also left behind a reusable framework for future research and a traceable record of open issues and support needs; that record informed the next joint-research phase and steered it toward a more theory-oriented direction.
