# Pipelines

Pipelines let you chain together data fitting, calculations, and validation
steps into a single server-side workflow. A pipeline is defined as an ordered
dictionary of named **elements**, each with an `element_type` and
type-specific configuration.

## Element types

| Type | Purpose |
|---|---|
| `entry` | Provide initial parameter values to downstream elements |
| `data_fit` | Fit model parameters to experimental data |
| `array_data_fit` | Fit the same model separately at each value of an independent variable (e.g. temperature, SOC) |
| `calculation` | Run a calculation (e.g. OCP fitting) |
| `validation` | Validate a model against experimental data |

Elements run in the order they appear. Later elements can reference results
from earlier ones through the `existing_parameters` field.

## Submitting a pipeline

### Entry element

An `entry` element seeds the pipeline with known parameter values:

```python
entry_config = {
    "element_type": "entry",
    "values": {
        "Negative particle diffusivity [m2.s-1]": 3.3e-14,
        "Positive particle diffusivity [m2.s-1]": 4e-15,
    },
}
```

### Data-fit element

A `data_fit` element optimizes model parameters against uploaded measurement
data. Reference measurement data stored in Ionworks with the `db:<id>` prefix:

```python
measurement_id = "..."  # from client.cell_measurement.create()

datafit_config = {
    "element_type": "data_fit",
    "objectives": {
        "test_1C": {
            "objective": "CurrentDriven",
            "model": {"type": "SPMe"},
            "data": f"db:{measurement_id}",
            "parameters": {
                "Ambient temperature [K]": 298.15,
            },
        },
    },
    "parameters": {
        "Negative particle diffusivity [m2.s-1]": {
            "bounds": [1e-14, 1e-13],
            "initial_value": 2e-14,
        },
        "Positive particle diffusivity [m2.s-1]": {
            "bounds": [1e-15, 1e-14],
            "initial_value": 2e-15,
        },
    },
    "cost": {"type": "RMSE"},
    "optimizer": {"type": "ScipyDifferentialEvolution"},
}
```

### Array-data-fit element

An `array_data_fit` element fits the same model separately at each key of the
`objectives` dictionary, where the key is the value of some independent variable
(e.g. temperature, midpoint stoichiometry, pulse SOC). It accepts the same
top-level fields as `data_fit`; the only difference is the shape of `objectives`:

```python
array_datafit_config = {
    "element_type": "array_data_fit",
    "objectives": {
        # key = value of the independent variable
        0.25: {"objective": "Pulse", "data": f"db:{m1_id}", ...},
        0.50: {"objective": "Pulse", "data": f"db:{m2_id}", ...},
        0.75: {"objective": "Pulse", "data": f"db:{m3_id}", ...},
    },
    "parameters": {
        "Positive particle diffusivity [m2.s-1]": {
            "bounds": [1e-15, 1e-13],
            "initial_value": 1e-14,
        },
    },
}
```

The result contains one fitted parameter value per independent-variable key, so
the fitted parameter is returned as a 2×N array (independent-variable values
along one row, fitted values along the other).

### Combining elements

Pass all elements as a dictionary to `client.pipeline.create()`:

```python
pipeline = client.pipeline.create(
    {
        "elements": {
            "entry": entry_config,
            "fit data": datafit_config,
        },
    }
)
print(f"Pipeline ID: {pipeline.id}")
```

## Waiting for results

Pipelines run asynchronously. `wait_for_completion` blocks until the pipeline
reaches a terminal status — `completed`, `failed`, or `canceled` — and raises
`IonworksError` if it failed or was cancelled:

```python
client.pipeline.wait_for_completion(pipeline.id, timeout=600)
result = client.pipeline.result(pipeline.id)
print("Fitted parameters:", result.element_results["fit data"])
```

Pass `raise_on_failure=False` to get the terminal submission back instead of
raising. If you do poll by hand, branch on `pipeline.is_terminal` and
`pipeline.is_failed` rather than comparing `pipeline.status` against a list of
statuses — the server decides which statuses are terminal, so a status added
later is classified correctly without an SDK upgrade.

## Listing pipelines

```python
# All pipelines
pipelines = client.pipeline.list()

# Filter by project
pipelines = client.pipeline.list(project_id="...", limit=10)
```

## Data sources

Pipeline elements can load data from several sources:

| Prefix | Description |
|---|---|
| `db:<measurement_id>` | Measurement stored in Ionworks |
| `file:<path>` | Local CSV file |
| `folder:<path>` | Folder of CSV files |

## Next steps

To run forward simulations instead of fitting to data, see {doc}`simulations`.