The purpose of this discussion was to identify frequent themes that were brought up in the four in-depth studies.
It is important to consider scaling issues. What are effective incentives of collaboratory participation, such as how to identify and attract domain experts to develop middleware. CFAR is an example where the urgency of the activity may have helped motivate people to participate. It is easier for new organizations to adopt the collaboratory technologies because they have nothing to "unlearn."
It is also important to consider the context of the data, when examining a shared instrument and people using data from that instrument. In Bugscope, students need to understand what they are looking at when they look at data. Metadata can help provide context and will give information such as the reliability of the data and when it was gathered.
There are issues that arise in developing middleware. If a collaboratory needs a programmer, it could get to be very expensive. There might be problems for an individual if they participate in multiple projects that each use different toolsets. He also pointed to the importance of investigating failures in order to identify the reasons behind failure.
There are a number of methodological issues that arise when studying collaboratories. It might be useful to code data at a low level and then build up. He pointed out that it is important to look across the collaboratories, so they can learn from each other. As an example, it might be interesting to compare Bugscope to UARC to examine the data archival process in UARC in order to identify practices that Bugscope could potentially develop and benefit from.
There is a distinction between active and passive resistance of technology. For instance, often people just choose not to use a particular piece of technology, which contrasts with when the use of technology is mandated and the resistance is very strong. There are different types of incentives to participate in a collaboratory. Some individuals might want to know the answer to the research questions, while others may value the importance, honor, and glory of telling others the answers that have been discovered. In thinking about collaboratories, these incentives play a role in whether a person is going to participate. The potential participants want to know whether the collaboratory will help uncover the answers. Once the answers are uncovered, questions arise with regards to what counts as contribution and who is credited with the discoveries.
There are management issues that can arise in collaboratories. It is often a challenge to get active group effort. Past studies on Industrial Districts could provide insights to the collaboratory world. In the Industrial Districts different firms bid for jobs that they don't have the capability to complete, and then they sub-contract the work to others.
The motivations for collaboratory formation include:
- access to resources,
- mandates to start one (e.g. by funders)
- facilitate an existing collaboration
- taking on a bigger problem, and
- education outreach.
Much of the success has been where collaboration already existed. A collaboratory often enables the collaboration and that pre-existing collaboration is not necessarily a pre-requisite.
It was important to identify the characteristics of a collaboratory that make people join and stay in a collaboratory. It could be a number of things, such as a strong leader or social network. There are parallels in the Virtual Community literature and reward systems, such as Slashdot's reputation points which relates this back to the motivation issues.
Do collaboration tools really matter? In the four in-depth case studies, tools were generally not mentioned as important. Is awareness information an underlying requirement. Discussion about awareness was not brought up much in the four case studies. How will cross-collaboratory collaboration be done and how will collaboratories be made aware of each other? This is one of the goals of the SOC project.
An alternative categorization scheme to the one presented in the Collaboratories-at-a-glance was presented. In this scheme, a collaboratory would not belong to just one category, but rather it would receive a percentage in each category. The categories are:
- Peer interactions,
- Scheduling coordination,
- Presentation to audience,
- Connection to operator,
- Connection to expert or instrument, and
- Guided investigations.
Conference participants noted that this classification scheme is not necessarily a replacement for the scheme presented in Collaboratory-at-a-glance, but rather just a different level of coding. This scheme might be good for use in tool development because it centers on the types of activities that people do. The test of a categorization scheme is whether it allows someone to make good recommendations.
Issues offered by workshop participants
It is often a challenge to motivate people to join collaboratories and it is important to identify the reasons people build and participate in a collaboratory.
There is a tension between collaboration and competition and the incentive structure for participation.
It is important to look at the evolution of collaboratories to garner best practices.
Things that should be examined include access, collaboration, integration, and the community throughout the lifecycle of the collaboratory.
CFAR, UARC, and EMSL are collaboratories that benefited from being flexible and were able to respond to needs that spontaneously arose.
How do we generalize findings from one collaboratory to other collaboratories. How is success determined? How do we standardize data collection procedures? How do we identify control groups?
We will want to consider which studies we do more in-depth analyses on. Which ones? What criteria? Some in the same category? Successes and failures? Ones with a lot of social data collected on them?
back to the Final Summary