ChenboLin
01About02Work03Labs04Notes
GET IN TOUCH
Back to engineering notes阅读中文版
Lab journal4 min read

From raw data to the next useful step.

Thoughts on automation workflows that do more than calculate — they help decide what to try next.

Published
2026-08-18
Updated
2026-08-18

Topics

AutomationPythonData provenanceEngineering workflow
01

A finished calculation is not a finished workflow

Automation is often described as replacing a manual calculation. That can save time, but the more important opportunity is to connect raw input to a decision that someone can inspect and act on.

A useful workflow therefore needs more than a final number. It should preserve where the input came from, which transformation was applied, what failed validation and which result is ready for the next step.

02

Keep inputs and methods next to the result

Files, parameters, software versions and intermediate outputs should remain traceable. If a result changes, the workflow should make it possible to distinguish a changed input from a changed method or an execution error.

This is especially important when a process spans scientific software, custom Python code and manually reviewed outputs. Reproducibility begins with making the path visible.

03

Decide when the workflow should stop

A robust workflow should stop when required data is absent, malformed or inconsistent. Silent fallbacks can make a pipeline look successful while weakening the evidence behind its output.

Clear validation messages and retained intermediate evidence turn failures into useful information. They show what needs attention instead of hiding the problem behind a completed progress bar.

04

Design the output around what happens next

The best output format depends on what happens next: comparison, review, another experiment or a physical test. Tables, plots and summaries should be selected to reduce the distance between the calculation and that next decision.

This is the standard I use for automation: not only whether it runs, but whether it helps a person understand what is known, what is uncertain and what is worth trying next.

Each page states what the project covers, what evidence is public and what has not yet been verified.

Browse all engineering notes
Chenbo Lin
© 2026 Chenbo Lin. All rights reserved.