Attivio Adds Transaction Like Document Processing
January 17, 2011
We learned that Attivio has added transaction like document processing to its Active Intelligence Engine. AIE is a scalable parallel asynchronous messaging system. Unlike other asynchronous systems methods, Attivio gives licensees controls to determine when a message is processed. Attivio reveals in “Transaction-like Document Processing in AIE”:
Grouped message processing allows a set (usually small) of related messages to be processed as a group (in order) when needed. This capability can be turned on or off by individual document processing transformers. What this means is that when the grouping isn’t required (the transformer doesn’t have side-effects which depend on the group ordering) then the messages are processed independently, delivering the maximum throughput. When a document transformer does require this transaction-like behavior, a simple configuration change is all that is necessary. This change causes the following semantics to come into play for the component:
- All non-document messages are blocked while processing the group. A system commit or optimize message cannot be processed until the group is complete. This means the processing of all the messages of the group will occur together before or after the commit, ensuring a consistent state.
- The documents in the group are sent to the component in the order they were added to the group.
- Only a single instance of the component is used to process the documents in the group.
The component that most heavily uses message groups is the ContentDispatcher (the gateway component for the AIE index).
You can get additional information, block diagrams, and links to special features on the Attivio Web site or in Attivio’s blog.
Stephen E Arnold, January 17, 2011
Freebie
Comments
-
- Subscribe to Beyond Search
Feature archive
News archive
-
