Generic CSV multi-import — GX-TXT Pipeline v0.8.6

Generic CSV multi-import — GX-TXT Pipeline v0.8.6

acq.generic_csv supports both the historical single-file path and a new multi-CSV batch table.

Single-file mode

The existing fields remain available and backward compatible:

Acquisition CSV
Canonical dataset ID
Dataset contract / kind
CSV delimiter
Header rows

The Channels/Variables page can still be used by the established single-file path to rename/map source columns.

Multi-CSV batch mode

In the Acquisition tab choose:

Import module = acq.generic_csv

The Multi-CSV canonical dataset import table appears below the ordinary configuration.

Controls:

Use batch table
+ Add CSV
Add multiple...
Clear
Contract registry...

Each enabled row contains:

Use
CSV file
Dataset ID
Contract
Other / custom contract
Delimiter
Header rows
Preview
Variables

Preview shows the selected file structure and sample rows before import. Variables opens the per-dataset variable map, where Original header and Canonical variable are displayed side by side.

The table has vertical and horizontal scrolling and is intended for an arbitrary practical number of input CSV files.

Add multiple files

Add multiple... allows selecting several files in one file-picker operation. GX-TXT creates one editable row per file and proposes a dataset ID from the filename.

Always review the proposed IDs before running the import.

Contract selection

Known contract IDs are supplied by the Contract Registry and therefore come from the framework plus installed plugin descriptors.

Current framework contracts include:

curve_family.table
raw.csv
table.generic
timeseries.table

If the correct contract is not listed, choose:

Other...

and type the exact contract in the adjacent field.

The custom value is stored literally in the session manifest and dataset registry.

Batch canonicalization

By default, batch mode uses the final configured header row as an identity mapping. Before import, each row may define an independent variable map. Canonical aliases are written into the canonical CSV header while RAW sources are preserved unchanged for provenance.

Example:

Original header -> Canonical variable
Vg1_V           -> Vg
Ib_mA           -> Ia
Ig1_mA           -> Ig

The resulting layout is conceptually:

session/
  raw/
    <dataset-id>/source.csv
  datasets/
    <dataset-id>.csv
  metadata/
    datasets.ini
    manifest.ini

Each dataset receives independent:

• ID;
• contract;
• source SHA-256;
• canonical SHA-256;
• canonical path;
• column list.

Example

electrical.csv  -> reactor.electrical -> timeseries.table
chemistry.csv   -> reactor.chemistry  -> table.generic
ftir_01.csv     -> ftir.sample01      -> Other... / spectrum.table

These datasets can then be consumed independently or jointly by analysis plugins and Scientific Analysis Definitions.

Manifest persistence

Batch configuration is stored under:

[generic_csv_batch]
api_version=1
enabled=1
count=...

[generic_csv_batch.1]
...

Custom/unknown contract values are preserved even if their original provider plugin is no longer available.

See also DATASET_INSPECTION_VARIABLE_MAPPING_API_v1.md for the normative mapping/provenance contract.

v0.8.7 — adding datasets after the session exists

The primary Import/Run action still creates the session and imports all enabled batch rows.

Once currentSession exists, the batch bar provides Import enabled pending into current session.

Eligibility is strictly:

Use = enabled
AND
Status = Pending

The incremental command never calls makeSession, never clears datasets.ini, and never imports disabled rows. Rows already registered by Dataset ID are shown as Imported and are not imported again.

acq.generic_csv

acq.generic_csv

Standard Module API v1 acquisition/import plugin for one or many generic CSV tables. It creates canonical datasets, assigns dataset IDs/contracts, snapshots RAW sources, and records SHA-256 provenance.

See also docs/guides/GENERIC_CSV_MULTI_IMPORT.md, docs/guides/MULTI_DATASET_IMPORT_AND_INSPECTION.md, docs/api/DATASET_API_v1.md, docs/api/DATASET_INSPECTION_VARIABLE_MAPPING_API_v1.md, and docs/api/CONTRACT_REGISTRY_API_v1.md.

Acquisition / Import

Acquisition / Import

Select an importer, add one or more files or a folder, inspect the pending queue, then import into the current Session. Successful items leave the queue; failed items retain their status for correction. The queue is an interactive work list and is not stored in the Session manifest.

Importer Builder creates reusable registered import definitions for tabular sources. Configure parsing, file matching, metadata and variable mapping against a representative file, test the definition, then select it in Acquisition. The definition is stored in the user configuration area, while every published Dataset records its importer identity and version.

Acquisition Handler plugins may also supply scientific or binary import paths. Their manuals describe expected file formats and interpretation options. GX-TXT preserves each source and records hashes and publication provenance.