Scalability really depends on the application. There are many COTS products out there for data indexing of unstructured data with varying scalability limits. Both Microsoft and Google have free offerings for the desktop. Google even has an Enterprise appliance offering. Solutions that are more business-centric come from Documentum and many others. I hadn’t heard of Njini before, but they seem to be focused on tailoring their software around business requirements also.

CAS is a complimentary technology that has other properties tied more to regulatory requirements. The “DIS” component creates the metadata indexes while CAS stores it securely, sometimes using de-duplication and/or WORM for storing the data.

All of these new applications have their own strengths and weaknesses - keeping in mind your specific requirements, you need to be diligent with your research (recommend evaluating the products you are interested in). Oh - and don’t get suckered into a really cool feature that you’ll never use! That never happens, right!

Good luck with your research! This market segment is still evolving, so it can be quite confusing.

Blog ya later!

Joe

Do you know any COTS (Commercial Off The Shelf) products or vendor names that are available today?

How scalable vertically and horizontally is the “DIS” solution?
Will it scale from the SOHO to the Enterprise? Enterprise only?
Will it scale from Desktops to HPC clusters? HPC only?

How is this different from a metadata solution like http://www.njini.com/
or CAS (Content Addressed Storage) solutions
or “tagging” solutions?
In the way it is Implemented?
For example, how does the “DIS” tell an application index file from an application data file?

I don’t expect an in-depth answer. I can dig that out for myself. Just a quick go-bye. I know, I could dig that out for myself too.