This repo is a companion to the Build AI-Powered Apps with OpenAI and Node.js course on Frontend Masters.
Document QA Query Function Lesson
A few of the Langchain methods used in this course have been deprecated. Here's an alternative approach:
Install the Langchain community module
npm i @langchain/community
Import the loaders
import { PDFLoader } from '@langchain/community/document_loaders/fs/pdf'
import { YoutubeLoader } from '@langchain/community/document_loaders/web/youtube'
import { CharacterTextSplitter } from 'langchain/text_splitter'
Create the loaders using the community methods:
In docsFromYTVideo
:
const loader = YoutubeLoader.createFromUrl(video, { language: 'en', addVideoInfo: true, })
const loadedDoc = await loader.load()
const splitter = new CharacterTextSplitter({
separator: ' ',
chunkSize: 2500,
chunkOverlap: 200,
})
return await splitter.splitDocuments(loadedDoc)
In docsFromPDF
:
const docsFromPDF = async () => { const loader = new PDFLoader('./xbox.pdf')
const loadedDoc = await loader.load()
const splitter = new CharacterTextSplitter({
separator: '. ',
chunkSize: 2500,
chunkOverlap: 200,
})
return await splitter.splitDocuments(loadedDoc)