Measurement loading¶

Options controlling how measurement data is selected and preprocessed before a fit sees it, and the payload shapes a measurement can be supplied in.

Schema for the options accepted when loading measurement data.

Bases: BaseSchema

Preprocessing steps applied to a measurement before a fit sees it.

Each step may also be given at the top level of the options bag, which is the older spelling; both are merged before the transforms run.

Parameters:

rest_to_ocp (bool, optional) – Reduce rest periods to an OCP curve.
sort (bool, optional) – Sort by the independent variable.
remove_duplicates (bool, optional) – Drop duplicated rows.
remove_extremes (bool, optional) – Drop the first and last points of an OCP curve.
filters (dict, optional) – Column filters to apply, keyed by column name.
interpolate (float or list of float, optional) – Resample onto this spacing or these points.
keep_first_ocp_point (bool, optional) – Keep the first point when reducing to an OCP curve. Ignored unless rest_to_ocp is set.

Examples

>>> iw.pipeline.DataLoaderTransforms(sort=True, remove_duplicates=True).to_config()
{'sort': True, 'remove_duplicates': True}

rest_to_ocp: bool | None¶

sort: bool | None¶

remove_duplicates: bool | None¶

remove_extremes: bool | None¶

filters: dict[str, Any] | None¶

interpolate: float | list[float] | None¶

keep_first_ocp_point: bool | None¶

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

Bases: BaseSchema

Options controlling how measurement data is loaded and preprocessed.

Selects a subset of the measurement and applies preprocessing transforms before a fit sees it. Every field is optional; unknown keys are rejected, so a misspelt option fails when you build the config rather than being silently ignored and changing your results.

Parameters:

transforms (dict, optional) – Preprocessing steps to apply, keyed by name — sort, remove_duplicates, remove_extremes, rest_to_ocp, keep_first_ocp_point, filters, interpolate. Each may also be given at the top level, which is the older spelling.
first_step (int, str, or dict, optional) – Where to start the slice: a step index, a Polars SQL query against the steps table, or the deprecated dict form ({"cycle": 3}, {"step": 7}).
last_step (int, str, or dict, optional) – Where to end the slice, in the same form as first_step.
capacity_column (str, optional) – Name of the column to treat as capacity when the default is not present.
first_step_dict (dict, optional) – Deprecated spelling of first_step.
last_step_dict (dict, optional) – Deprecated spelling of last_step.
rest_to_ocp (bool, optional) – Reduce rest periods to an OCP curve.
sort (bool, optional) – Sort by the independent variable before use.
remove_duplicates (bool, optional) – Drop duplicated rows.
remove_extremes (bool, optional) – Drop the first and last points of an OCP curve.
filters (dict, optional) – Column filters to apply, keyed by column name.
interpolate (float or list of float, optional) – Resample onto this spacing or these points.
keep_first_ocp_point (bool, optional) – Keep the first point when reducing to an OCP curve. Ignored unless rest_to_ocp is set.

Examples

>>> options = iw.pipeline.DataLoaderOptions(first_step=2, transforms={"sort": True})
>>> options.to_config()
{'transforms': {'sort': True}, 'first_step': 2}

transforms: DataLoaderTransforms | None¶

first_step: int | str | dict[str, Any] | None¶

last_step: int | str | dict[str, Any] | None¶

capacity_column: str | None¶

first_step_dict: int | str | dict[str, Any] | None¶

last_step_dict: int | str | dict[str, Any] | None¶

rest_to_ocp: bool | None¶

sort: bool | None¶

remove_duplicates: bool | None¶

remove_extremes: bool | None¶

filters: dict[str, Any] | None¶

interpolate: float | list[float] | None¶

keep_first_ocp_point: bool | None¶

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

class ionworks.pipeline.data_loader.TimeSeriesSpec(*, time_series: Any, steps: Any = None)[source]¶

Bases: BaseSchema

A measurement given as its time-series table, optionally with steps.

Parameters:

time_series (Any) – The time-series table, as a DataFrame or a column mapping.
steps (Any, optional) – The step table, in the same form.

time_series: Any¶

steps: Any¶

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

class ionworks.pipeline.data_loader.TimeRangeSpec(*, start: float, end: float)[source]¶

Bases: BaseSchema

A window of elapsed seconds restricting which samples a source yields.

Bounds are Time [s] values, never wall-clock: start_time is editable metadata a pin must not silently re-anchor to. Give the far bound as end or duration, which resolves to end at construction. Step selection comes first: first_step picks from the whole measurement, and the window narrows within the steps it chose.

Parameters:

start (float) – Inclusive lower bound, in seconds from the first sample.
end (float) – Inclusive upper bound, in seconds. Must not precede start. Supply this or duration, not both.
duration (float) – Seconds of data to use, measured from start. Input only: it sets end and is not itself part of the window.

Examples

>>> TimeRangeSpec(start=0.0, end=3600.0).to_config()
{'start': 0.0, 'end': 3600.0}

The first day of a test, however long that test eventually runs:

>>> TimeRangeSpec(start=0.0, duration=86_400.0).to_config()
{'start': 0.0, 'end': 86400.0}

start: float¶

end: float¶

classmethod resolve_duration(data)[source]¶

Turn a duration into the end the wire format carries.

Resolved before validation rather than kept as a field so duration never reaches the config: consumers read start/end and reject anything else, and a window storing both could disagree with itself after an assignment.

validate_ordering()[source]¶: Reject a window that ends before it starts.

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

class ionworks.pipeline.data_loader.AnalysisSpec(*, analysis_id: str | None = None, analysis_type: str | None = None, name: str | None = None)[source]¶

Bases: BaseSchema

Selector naming one analysis already stored on a measurement.

An analysis is a stored artifact, not something computed on demand: a row in the platform’s analyses table plus a parquet of extracted features. This selector says which of a measurement’s analyses to read as the fit’s target data. Every field given must match, so two analyses sharing a type are separated by name.

Parameters:

analysis_id (str, optional) – The analysis’ own id. Exact, and the form a resolved run records, so a re-run cannot drift onto a different analysis.
analysis_type (str, optional) – Kind of analysis, e.g. "lam_lli_from_rpt". Free-form on the platform, so any string is accepted.
name (str, optional) – The analysis’ name, as given when it was created.

Examples

>>> iw.pipeline.AnalysisSpec(analysis_type="lam_lli_from_rpt").to_config()
{'analysis_type': 'lam_lli_from_rpt'}

analysis_id: str | None¶

analysis_type: str | None¶

name: str | None¶

validate_selector_is_not_empty()[source]¶: Reject a selector that would match every analysis on the measurement.

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

class ionworks.pipeline.data_loader.DataPayloadSpec(*, data: Any, options: DataLoaderOptions | None = None, time_range: TimeRangeSpec | None = None, analysis: AnalysisSpec | None = None)[source]¶

Bases: BaseSchema

A measurement plus the options controlling how it is loaded.

Parameters:

data (Any) – The measurement itself — a path spec, a table, or a nested payload.
options (DataLoaderOptions, optional) – How to slice and preprocess it. Unknown keys are rejected.
time_range (TimeRangeSpec, optional) – Elapsed-seconds window pinning a db: source to the samples it held when the run was first pinned, so a re-run is unaffected by a later extend.
analysis (AnalysisSpec, optional) – Read one of the measurement’s stored analyses as the data, rather than its time series. Cannot be combined with options or time_range, neither of which can apply to a feature table.

data: Any¶

options: DataLoaderOptions | None¶

time_range: TimeRangeSpec | None¶

analysis: AnalysisSpec | None¶

validate_analysis_stands_alone()[source]¶

Reject options that cannot apply to an analysis table.

options are DataLoader options, and an analysis resolves to a bare frame that never reaches a DataLoader, so they would be silently ignored. time_range is a window of elapsed seconds into a time series, which a per-cycle feature table does not have.

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.

class ionworks.pipeline.data_loader.MetadataSpec(*, data: Any, metadata: dict[str, Any])[source]¶

Bases: BaseSchema

A measurement carried alongside its provenance metadata.

Parameters:

data (Any) – The measurement itself.
metadata (dict) – Provenance recorded when the measurement was loaded.

data: Any¶

metadata: dict[str, Any]¶

model_config = {'arbitrary_types_allowed': True, 'extra': 'forbid', 'populate_by_name': True, 'validate_assignment': True}¶: Configuration for the model, should be a dictionary conforming to [ConfigDict][pydantic.config.ConfigDict].

model_post_init(context: Any, /) → None¶

This function is meant to behave like a BaseModel method to initialize private attributes.

It takes context as an argument since that’s what pydantic-core passes when calling it.

Args:: self: The BaseModel instance. context: The context.