chunking_none()

Disable chunking for one-to-one embedding relationships

Disable chunking to maintain a one-to-one relationship between source rows and embeddings. This is required when using column destination, where embeddings are stored directly in the source table.

When to use

Use chunking_none() when:

• Using destination_column() (required)
• Source text is already chunked or naturally short (< 512 tokens)
• You need exactly one embedding per source row

Samples

With column destination (required)

SELECT ai.create_vectorizer(
    'products'::regclass,
    destination => ai.destination_column('description_embedding'),
    loading => ai.loading_column('description'),
    embedding => ai.embedding_openai('text-embedding-3-small', 768),
    chunking => ai.chunking_none()  -- Required for column destination
);

With pre-chunked data

SELECT ai.create_vectorizer(
    'pre_chunked_text'::regclass,
    loading => ai.loading_column('chunk'),
    embedding => ai.embedding_openai('text-embedding-3-small', 768),
    chunking => ai.chunking_none()
);

Arguments

This function takes no parameters.

Returns

A JSON configuration object for use in create_vectorizer().

Related functions

• destination_column(): requires chunking_none()
• chunking_character_text_splitter(): split by character count
• chunking_recursive_character_text_splitter(): recursive splitting