Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature] Repair JSON from LLM in StructuredOutputParserAdvanced #3723

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

JJK801
Copy link
Contributor

@JJK801 JJK801 commented Dec 17, 2024

Hi Flowise,

I got in trouble with using StructuredOutputParserAdvanced with some complex schema: LLM returns broken JSON (wrong quotes, double quote surrounding, etc...)

I added a lib that fixes JSON text before trying to parse it.

@JJK801
Copy link
Contributor Author

JJK801 commented Dec 17, 2024

Results are quite impressive, from arround 10% success rate, i now got 100% success (or close to, i didn't have any formatting error since)

@HenryHengZJ
Copy link
Contributor

interesting, can you give an example of schema that you were using?

@JJK801
Copy link
Contributor Author

JJK801 commented Dec 18, 2024

Of course, here is a Zod schema we use for generating a report after a chat session with a client:

z.object({
	title: z.string().describe('A title for the report'),
	type: z
		.string()
		.describe(
			'Type of intervention - e.g., inspection, quality control, troubleshooting, training'
		),
	overview: z.string(),
	steps: z
		.array(z.string())
		.describe('Detailed description of the steps taken during the intervention'),
	summary: z.object({
		short: z
			.string()
			.describe(
				'Short summary of the intervention, mention the name of the user if you have the information'
			),
		long: z.string().describe('Long summary of the intervention')
	}),
	local: z.object({
		totalSteps: z
			.number()
			.describe(
				'Total number of steps in the first level, not counting the substeps, in the whole Procedure'
			),
		stepsCompleted: z
			.number()
			.describe('Steps completed in the first level, not counting the completed substeps')
	}),
	issuesEncountered: z
		.array(z.string())
		.describe('List of issues encountered during the intervention'),
	finalOutcome: z.string().describe('Final outcome of the intervention'),
	tasks: z.object({
		toDo: z
			.array(z.string())
			.describe('Tasks emerging from the intervention that need to be addressed'),
		done: z.array(z.string()).describe('Tasks completed during the intervention'),
		suspended: z.array(z.string()).describe('Pending tasks')
	}),
	sentiments: z.object({
		client: z.array(z.string()).describe('Sentiment analysis of the user'),
		hotliner: z.array(z.string()).describe('Sentiment analysis of the hotliner')
	}),
	machine: z.object({
		type: z.string().describe('Type of machine involved - if applicable, if not then empty')
	})
})

Most of the time, the 5~8 firsts keys of the JSON are valid, but the other have errors, mainly using simple quotes or curly quotes.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants