Banzai is a document processing framework for Node.js.

You define a set of pipelines into which you push documents. Each document in a pipeline has a given state. A state transition triggers a state entry handler that can transform the document and interact with the outside world. The documents ends in a defined or in an “error” state.

Rollback and Playback

You can roll-back the state of a document to a certain previous state, and playback the pipeline flow. This can be useful, for instance, if a given document enters an error state because of a bug or a networking problem somewhere. You can correct the bug, roll-back to a previous state and play the pipeline from thereon, hopefully escaping that error condition.

Each state transition has a “next state”, a priority and a optional pre-condition. The candidate transitions (there can be more than one) are evaluated in the given priority, and if there is a pre-condition, it is evaluated, and if there is a match, the corresponding state transition handler is triggered.

Each state transition can have an “undo handler”, that takes care of undoing the document. This can be useful if external services were changed and you need to revert those changes when you revert a transition.

All JavaScript

The state transition handlers and the pre-conditions are all defined in JavaScript and are asynchronous, meaning that you can perform I/O inside them. The pipeline definition is also written in JavaScript.

Architecture

A Banzai deployment has 4 main components: the document store, the state store, the workers and the work queue.

The Document Store

The document store is where - you guessed it - the documents are stored. The document is retrieved when entering a state transition, passed into the state transition handler and saved when the handler is done. This way a state transition can be picked by any worker and the document is always persisted, surviving failures.

State Store

The state store is where the state for each document transitioning or that has transitioned a pipeline is stored. There you can also find some additional meta-data, like all the transitions that occurred and their start and end times plus some meta-data that the state transitions can save.

Workers

Workers are processes that are listening for state transitions and that pick up the work of invoking the state transition handler and deciding the next state.

Work Queue

The Work Queue is an event queue that persists and distributes the transitions to be picked up by the workers.

Adapters

Doc Store and State Store

Currently the only supported database is CouchDB, but technically any document database is supported. It should, by the way, store every version of the documents (as CouchDB does) if you want to be able to roll-back to certain versions of the documents.

The module for supporting CouchDB is banzai-couchdb-store.

Queue

Currently we support Redis (any version >= 2.1.7) if you use the banzai-redis module, but any queueing system that allows the same semantics should work.

Show me the code!

Check out the project README.

Tags:   node.js, javascript, couchdb,
Notes
  1. wactbprot reblogged this from metaduck
  2. metaduck posted this