`dyn` and pipelines #13

New issue

Open

opened 2024-02-26 18:16:21 +00:00 by multisamplednight · 0 comments

multisamplednight commented

2024-02-26 18:16:21 +00:00

Collaborator

As part of #12, @schrottkatze is implementing an extensible pipeline system, in a library named prowocessing. However, this extensibility should be possible for both instructions (named "pipeline elements" in #12) and data types.

@schrottkatze brought up that going full dyn would not be undesirable, since that would make

pipelines susceptible to incorrect input data
it imposssible to emit a useful error for this incorrect input data
since type checks have to happen at runtime as a result

"runtime" here is ambiguous. It might refer to

the program runtime of a host using prowocessing
the iOwO stage of runtime, which at the moment is planned to consist out of scheduler and evaluator

In the first interpretation, I agree with 3., but not with 1. and 2. There can still be a validation stage before actual execution, which could check if the output signatures of outgoing neighbors in the DAG match the input signatures of incoming neighbors.
The functions for validation and actual runtime are not limited in terms of return type, and may as well return a proper Result with a proper error type.

In the second interpretation, I don't agree with any of these, as listed above already the type safety can still be ensured in the typecheck stage before it's passed into the optimizer.

As part of #12, @schrottkatze is implementing an extensible pipeline system, in a library named `prowocessing`. However, this extensibility should be possible for ***both*** instructions (named "pipeline elements" in #12) and data types. @schrottkatze brought up that going full `dyn` would not be undesirable, since that would make 1. pipelines susceptible to incorrect input data 2. it imposssible to emit a useful error for this incorrect input data 3. since type checks have to happen at runtime as a result "runtime" here is ambiguous. It might refer to - the program runtime of a host using `prowocessing` - the iOwO stage of runtime, which at the moment is planned to consist out of scheduler and evaluator In the first interpretation, I agree with 3., but not with 1. and 2. There can still be a validation stage before actual execution, which could check if the output signatures of outgoing neighbors in the DAG match the input signatures of incoming neighbors. The functions for validation and actual runtime are not limited in terms of return type, and may as well return a proper `Result` with a proper error type. In the second interpretation, I don't agree with any of these, as listed above already the type safety can still be ensured in the typecheck stage before it's passed into the optimizer.

No labels

proposal

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: katzen-cafe/iowo#13

No description provided.

dyn and pipelines #13

`dyn` and pipelines #13