Information Retrieval and Collaboration Tools

A *wonderful* mini-essay, Arun! I especially like a couple of terms you use, which we perhaps should carry forward in our evolving discussion vocabulary. AgentWorlds/ServiceLands. I suspect that we will have more so say about this interaction space as we move forward with our visioning, reference architectures and implementations.

I have been mulling a few other thoughts and issues that are not fully formed yet, but should enter into our mix of discussion...

- I am becoming increasingly convinced that a large number of high-powered, supporting technologies in areas of Information Retrieval (IR), meta-information creation and manipulation/management and semantic web support will be essential to our vision and reference development directions.

I am now moving onto a new project in which I will be responsible for portions of UI and infrastructure to support a "showcase" for current and evolving technologies that our department is proficient in and working on. A growing number of these technologies are (and will continue to be) related to information retrieval (IR) and knowledge engineering/inference from data and meta-data made available by IR and subsequent text analytic tasks. My early research into available tools to support these technologies indicates that there are many now, and the number is growing rapidly (see LingPipe and its very nice summary page of this product's perceived "competition" at: http://www.alias-i.com/lingpipe/web/competition.html).

*** Advocate agents will absolutely need to work in this problem space as part of their (likely) insatiable need for dynamically/continuously updated (meta-)data/information/knowledge to support their behaviors.

Current directions and tools in support of the Semantic Web nicely dovetail with these technologies, and will also form an essential part of a reasonable "advocate agent technology stack". See the current MIT work on the SIMILE projects (http://simile.mit.edu/) for (what I consider to be) exciting "teases" of the necessary directions in which we must go to realize a powerful vision of human assistance by advocate agents. As Wolf has intimated, these directions will most likely need to address (as easy as possible) "mashup-ability" of features, content and meta-information.

We need to include and pragmatically limit these considerations in our vision and directions.

- Focusing a bit more on the "pragmatics" I mention above ... It would be extremely easy to create a grand vision that would be highly desirable, but impossible to meet in a realistic manner in any reasonable timeframe. I suggest that we a) articulate and maintain an evolving/living grand vision statement, but, to make our efforts see continuing value, b) set realistic time-boxed goals that incrementally move the effort forward in meaningful ways.

I strongly suggest that these time-boxes be no longer than 6 months in length, with both demonstration and documentary deliverables at the end of each of these cycles. I suspect that there would typically be several agile development iterations within each of these larger goal-cycles.

Some suggested process to guide our efforts:

The above approach would give our project efforts some structure. Our first "goal cycle" would include a) development and publication of a first version of our (living) "big picture" vision, b) development and publication of a set of goals for the next (first) 6-month project iteration.

We could then flesh this out with a (very) few agile, all-hands planning sessions to determine the a) prioritized "vision" backlog of tasks, b) prioritized backlog of stories and tasks for the NEXT goal-cycle (including work assignments to specific persons). The stories, tasks and priorities will be updated during regular meetings, and specifically during start-of-iteration planning meetings. We should hold retrospectives on the process at least every goal-cycle. Goal cycles will end with demonstrations of substantive results and publication of supporting documentation and related scholarly papers, internally and to relevant conferences.

If there are no objections, I suggest we apply a variant of Scrum to do this and guide our process steps and process improvement. We may or may not want to use a tool such as the freely available ScrumWorks Basic (http://danube.com/scrumworks/basic) to help us keep track of our "burndown" or "burnup" and productivity "velocity" statistics.

We will need group collaboration tools. I am open to suggestions here, but I think that perhaps Skype (to support free international telecon) plus a collaborative meeting/appSharing tool will be needed. NetMeeting will most likely work. More than a handful of others are certainly available. (Any recommendations? - don't forget to explain why!)

We have an opportunity to use the Qwaq forum product (www.qwaq.com) to good advantage here, if we are willing and able to spend ~$50 per person per month for persistent cyberspace access to one or more virtual planning and meeting "rooms" devoted to advocate agent projects (perhaps running in parallel). This service provides VOIP + collaboration support in virtual world settings, so we might be able to do without Skype if we go this route. What do others think?

-----

Comments?

-Gary