Publications
Search
1.
Addison, Parker; Nguyen, Minh-Tuan H.; Medan, Tomislav; Shah, Jinali; Manzari, Mohammad T.; McElrone, Brendan; Lalwani, Laksh; More, Aboli; Sharma, Smita; Roth, Holger R.; Yang, Isaac; Chen, Chester; Xu, Daguang; Cheng, Yan; Feng, Andrew; Xu, Ziyue
C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System Miscellaneous
2024, (arXiv:2412.13163 [cs]).
@misc{addison_c-fedrag_2024,
title = {C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System},
author = {Parker Addison and Minh-Tuan H. Nguyen and Tomislav Medan and Jinali Shah and Mohammad T. Manzari and Brendan McElrone and Laksh Lalwani and Aboli More and Smita Sharma and Holger R. Roth and Isaac Yang and Chester Chen and Daguang Xu and Yan Cheng and Andrew Feng and Ziyue Xu},
url = {http://arxiv.org/abs/2412.13163},
doi = {10.48550/arXiv.2412.13163},
year = {2024},
date = {2024-12-01},
urldate = {2025-03-20},
publisher = {arXiv},
abstract = {Organizations seeking to utilize Large Language Models (LLMs) for knowledge querying and analysis often encounter challenges in maintaining an LLM fine-tuned on targeted, up-to-date information that keeps answers relevant and grounded. Retrieval Augmented Generation (RAG) has quickly become a feasible solution for organizations looking to overcome the challenges of maintaining proprietary models and to help reduce LLM hallucinations in their query responses. However, RAG comes with its own issues regarding scaling data pipelines across tiered-access and disparate data sources. In many scenarios, it is necessary to query beyond a single data silo to provide richer and more relevant context for an LLM. Analyzing data sources within and across organizational trust boundaries is often limited by complex data-sharing policies that prohibit centralized data storage, therefore, inhibit the fast and effective setup and scaling of RAG solutions. In this paper, we introduce Confidential Computing (CC) techniques as a solution for secure Federated Retrieval Augmented Generation (FedRAG). Our proposed Confidential FedRAG system (C-FedRAG) enables secure connection and scaling of a RAG workflows across a decentralized network of data providers by ensuring context confidentiality. We also demonstrate how to implement a C-FedRAG system using the NVIDIA FLARE SDK and assess its performance using the MedRAG toolkit and MIRAGE benchmarking dataset.},
note = {arXiv:2412.13163 [cs]},
keywords = {},
pubstate = {published},
tppubtype = {misc}
}
Organizations seeking to utilize Large Language Models (LLMs) for knowledge querying and analysis often encounter challenges in maintaining an LLM fine-tuned on targeted, up-to-date information that keeps answers relevant and grounded. Retrieval Augmented Generation (RAG) has quickly become a feasible solution for organizations looking to overcome the challenges of maintaining proprietary models and to help reduce LLM hallucinations in their query responses. However, RAG comes with its own issues regarding scaling data pipelines across tiered-access and disparate data sources. In many scenarios, it is necessary to query beyond a single data silo to provide richer and more relevant context for an LLM. Analyzing data sources within and across organizational trust boundaries is often limited by complex data-sharing policies that prohibit centralized data storage, therefore, inhibit the fast and effective setup and scaling of RAG solutions. In this paper, we introduce Confidential Computing (CC) techniques as a solution for secure Federated Retrieval Augmented Generation (FedRAG). Our proposed Confidential FedRAG system (C-FedRAG) enables secure connection and scaling of a RAG workflows across a decentralized network of data providers by ensuring context confidentiality. We also demonstrate how to implement a C-FedRAG system using the NVIDIA FLARE SDK and assess its performance using the MedRAG toolkit and MIRAGE benchmarking dataset.
Filter
2024
Addison, Parker; Nguyen, Minh-Tuan H.; Medan, Tomislav; Shah, Jinali; Manzari, Mohammad T.; McElrone, Brendan; Lalwani, Laksh; More, Aboli; Sharma, Smita; Roth, Holger R.; Yang, Isaac; Chen, Chester; Xu, Daguang; Cheng, Yan; Feng, Andrew; Xu, Ziyue
C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System Miscellaneous
2024, (arXiv:2412.13163 [cs]).
Abstract | Links | BibTeX | Tags: and Cluster Computing, Computer Science - Distributed, Computer Science - Information Retrieval, Parallel
@misc{addison_c-fedrag_2024,
title = {C-FedRAG: A Confidential Federated Retrieval-Augmented Generation System},
author = {Parker Addison and Minh-Tuan H. Nguyen and Tomislav Medan and Jinali Shah and Mohammad T. Manzari and Brendan McElrone and Laksh Lalwani and Aboli More and Smita Sharma and Holger R. Roth and Isaac Yang and Chester Chen and Daguang Xu and Yan Cheng and Andrew Feng and Ziyue Xu},
url = {http://arxiv.org/abs/2412.13163},
doi = {10.48550/arXiv.2412.13163},
year = {2024},
date = {2024-12-01},
urldate = {2025-03-20},
publisher = {arXiv},
abstract = {Organizations seeking to utilize Large Language Models (LLMs) for knowledge querying and analysis often encounter challenges in maintaining an LLM fine-tuned on targeted, up-to-date information that keeps answers relevant and grounded. Retrieval Augmented Generation (RAG) has quickly become a feasible solution for organizations looking to overcome the challenges of maintaining proprietary models and to help reduce LLM hallucinations in their query responses. However, RAG comes with its own issues regarding scaling data pipelines across tiered-access and disparate data sources. In many scenarios, it is necessary to query beyond a single data silo to provide richer and more relevant context for an LLM. Analyzing data sources within and across organizational trust boundaries is often limited by complex data-sharing policies that prohibit centralized data storage, therefore, inhibit the fast and effective setup and scaling of RAG solutions. In this paper, we introduce Confidential Computing (CC) techniques as a solution for secure Federated Retrieval Augmented Generation (FedRAG). Our proposed Confidential FedRAG system (C-FedRAG) enables secure connection and scaling of a RAG workflows across a decentralized network of data providers by ensuring context confidentiality. We also demonstrate how to implement a C-FedRAG system using the NVIDIA FLARE SDK and assess its performance using the MedRAG toolkit and MIRAGE benchmarking dataset.},
note = {arXiv:2412.13163 [cs]},
keywords = {and Cluster Computing, Computer Science - Distributed, Computer Science - Information Retrieval, Parallel},
pubstate = {published},
tppubtype = {misc}
}
Organizations seeking to utilize Large Language Models (LLMs) for knowledge querying and analysis often encounter challenges in maintaining an LLM fine-tuned on targeted, up-to-date information that keeps answers relevant and grounded. Retrieval Augmented Generation (RAG) has quickly become a feasible solution for organizations looking to overcome the challenges of maintaining proprietary models and to help reduce LLM hallucinations in their query responses. However, RAG comes with its own issues regarding scaling data pipelines across tiered-access and disparate data sources. In many scenarios, it is necessary to query beyond a single data silo to provide richer and more relevant context for an LLM. Analyzing data sources within and across organizational trust boundaries is often limited by complex data-sharing policies that prohibit centralized data storage, therefore, inhibit the fast and effective setup and scaling of RAG solutions. In this paper, we introduce Confidential Computing (CC) techniques as a solution for secure Federated Retrieval Augmented Generation (FedRAG). Our proposed Confidential FedRAG system (C-FedRAG) enables secure connection and scaling of a RAG workflows across a decentralized network of data providers by ensuring context confidentiality. We also demonstrate how to implement a C-FedRAG system using the NVIDIA FLARE SDK and assess its performance using the MedRAG toolkit and MIRAGE benchmarking dataset.