Sub ledger consolidation program using Microgen Aptitude
A few years ago, I worked in the past with a niche product called OST Business Rules, which empowers finance business users to develop their own accounting engine rules in a nice user friendly manner, therefore reducing the need for heavy development works from Finance IT.
OST was subsequently bought by Microgen, and where OST Business Rules have now evolved to a product called Microgen Aptitude. The concept is still fairly similar. You take various feeds from your upstream systems, either in the form of flat files, or xml messages. Aptitude via standard loader or MQ, will load the data into the database, and enrich them, prior to applying all the required business rules, to feed into various target, such as the general ledger, financial repository (data warehouse), for reporting purposes.
While Microgen Aptitude feels like a godsend to accountants, which allows them to create and implemented accounting business rules relatively quickly, like any other automatic wizzardy type applications, it can be misused, and create a maintenance nightmare.
Microgen Aptitude requires a database server and an application server.
The key point here is that data is processed row by row.
In order to achieve high throughput, the applications relies on sufficient bandwith between the database server and the application server, as well as relies heavily on concurrent process to be able to increase its throughput. So lets say you have 60k accounting events to process, and you have 1 hour to process it, that means will need to process 1000 events per minute. Depending on complexity of required enrichment, plus the complexity of the accounting rules, this could prove to be a lot of events to process within a minute. In this case you will perhaps be running 10 streams in parallel.
So yes, network traffic will be quite busy. You will be getting hot blocks on your tables, as multiple processes are likely to want to write to the same block at the same time.
To be continued …
I am pleased to update that Microgen Aptitude doesnt need a Database and it doesnt need an App Server, these are implementation choices. It is no longer row based processing unless you want it to be. We have data in Oracle Coherence and connect to that data through a Java API in Aptitude. Its very scalable and we only see Network IO. We have some issues around manageablity but we are working those through. It is a very memory hungry/inefficient thing however.
PS - Microgen appear to be optimising Aptitude for use with Terra Data. We are pushing for a Oracle Exadata version.
Row by row processing is nothing special for, let’s say an Oracle database - any PL/SQL procedure or engine will processing this manner unless it is set based SQL. Oracle is quite comfortable in processing multiple updates or inserst to the same table (the output of any accounting engine is typically inserted CRs/DRs anyway so updated ar not an issue). As far as subsequent building of the sub-ledger balance is concerned, it is quite possibel to segregate these to reduce block contention in the underlying technical environment which is being used.
Aptitude is a very powerful engine of this type and very good for analyst formaulation and understanding of business/accounting rules - technically we have proven that it can be as fast as Java code and it does not, in itself, required a relational database, only if you want to read and/or write data to one as source/target of your rules, but it is quite possible to run it in an all-memory environment (I’m currently working with it in an Oracle Coherence/Java distributed memory architecture).
It’s worth noting that Microgen, whilst they sell Aptitude as a standalone product, have also encapsulated it within a higher-functionality product called MAH (Microgen Accounting Hub) which is being increasingly adopted across a wide investment and retail bank client base as well as in other sectors such as telecoms and digital media. It provides an accounting engine, sub-ledger and data warehouse (latter on Oracle or Teradata).
I don’t work for Microgen nor am I affiliated with them (any more) and have worked with theit products for a number of years.
Thanks for the updates from the new Micogen world, guys