How to Convert CSV Data into JSONL for AI Training

Define a record schema first

JSONL stores one JSON value per physical line; this guide uses one object per record. It is a serialization convention, not a guarantee that a particular model accepts your fields. Decide whether each row is a record, a message or a complete conversation. A HubSpot record export is not automatically a dialogue, and Salesforce associations require relationship preparation before conversion.

Keep IDs and decimal strings intact unless the schema explicitly requires conversion. Decide how empty strings, missing values and source null markers map. Never turn every blank into zero or every numeric-looking ID into a number. Store timestamp timezone meanings separately from the fact that a value is text.

Parse CSV with a real parser

The Python CSV library handles quoted delimiters and embedded newlines; open CSV files with newline=''. Its reader returns strings by default. The JSON library safely escapes quotes and newlines. Do not build JSON by concatenating source text or split CSV using commas.

The following Python 3 example expects exactly record_id,text. It is intentionally an allowlisted intermediate schema, not a platform-specific export converter. Adapt and review the field mapping for your source. Run it in a restricted working directory with input named reviewed.csv; output uses exclusive creation so an existing candidate is not overwritten.

import csv
import json
from pathlib import Path

source = Path("reviewed.csv")
target = Path("candidate.jsonl")
expected = ["record_id", "text"]
count = 0
try:
    with source.open(encoding="utf-8-sig", newline="") as src:
        reader = csv.DictReader(src, strict=True)
        if reader.fieldnames != expected:
            raise ValueError("Expected unique headers: record_id,text")
        with target.open("x", encoding="utf-8", newline="\n") as dst:
            for row in reader:
                if None in row or any(v is None for v in row.values()):
                    raise ValueError(f"Malformed CSV near line {reader.line_num}")
                if not row["record_id"].strip():
                    raise ValueError("Missing record_id")
                dst.write(json.dumps(row, ensure_ascii=False, allow_nan=False) + "\n")
                count += 1
except Exception:
    # A partial candidate must remain quarantined, not released.
    raise
print(f"Converted {count} records; validate before release")

The code leaves a partial candidate on failure so it can be investigated in the restricted workspace; quarantine it and rerun to a fresh path after correcting the source. The script does not redact personal data, enforce unique record IDs or group messages into conversations. Add those task-specific checks before release.

Validate the converted file

Test a synthetic row with a leading-zero ID, a comma, quotation marks, non-English text and a multiline body. Confirm the ID stays a string and the decoded text is unchanged. The embedded newline becomes an escaped character in the JSON line; it must not create another physical record.

Parse every line, validate required fields and types, and reconcile record counts. Check duplicate IDs separately. If the input uses a different delimiter or encoding, configure and test it explicitly rather than assuming this example fits. Large text fields may need a reviewed CSV field-size setting; do not swallow parse errors.

Assemble task examples after conversion

For a conversation, collect ordered messages under a conversation key and preserve speaker roles, parent references and completeness flags. For structured records, retain relationship keys and a schema dictionary. A generic text object is often an intermediate format, not the final training contract.

Write checksums, counts, schema version and conversion settings into the manifest. Keep source-to-output lineage restricted if it contains identifying references.

Troubleshoot and assess readiness

Why are there more lines than CSV records? A CSV record may span lines. Count parsed records, not text-editor lines.

Why did my output lose leading zeros? Inspect earlier spreadsheet imports and implicit type conversions; the parser cannot recover values already changed upstream.

Conversion helps interoperability. Quality, authorized use and task fit remain separate requirements for AI licensing or internal model development.