RAG using files (pdf, html, csv etc.)

[deepinderdeol]

How can files like pdf, html, csv etc. be used for RAG?

I tried this with LlamaIndex where all files in a directory are loaded and vector index can be created/persisted. You can then query the index asking questions about content in the files. I haven't tried Kernel-Memory and looks like it could do something similar, but my understanding is that KM is going to remain an experimental project.

So are there any other options for this or will this be possible with Semantic-Kernel?

[sophialagerkranspandey]

Currently Semantic Kernel is specializing in providing real time tools to agents. We don't have anything in our roadmap to ingest offline data. Instead we recommend using AzureAI Search to ingest documents.

[goyaweb]

You can try Kernel Memory. Works well for that purpose!
https://github.com/microsoft/kernel-memory

[deepinderdeol]

Thanks. Yes, Kernel Memory should work, but it will remain an experimental project.

[John-Bell]

On another thread, I was told between SK memory and kernel memory, SK memory had a future.

Any reason we can’t just use SK memory for the retrieval part?

[sorin-costea]

I used Azure Document Intelligence for that part