bds pipelines

Data pipelines

bds combines the simplicity of scripting and the power to run on any computer. After all, a laptop, a cluster, a cloud and a datacenter are just computers. You should not need different frameworks to run program on each of them.

Develop

Develop

No need to spin instances or wait for cluster queues while developing and debugging. The same script runs without changes on laptops, clusters, or cloud. So you can develop pipelines faster and easier. Turn arround times are faster, debug times are shorter.

Run

Run

Once your script is ready, you just run it ont the cloud. No need to change the script. Coordinating jobs, spinning instances, logging, all done transparently.

User's group

User's Discussions

You can ask questions, coding tips in the new discussions.

User's group

Paper

Read our paper [pdf] for a detailed introduction. Keep in mind that this paper only covers bds version 0.9, and many features and improvements have been added since. If you use bds in an academic environment, please cite our work.

Syntax highlight

Edit

Edit bds programs using the VI syntax highlight



View documentation »