Academia.eduAcademia.edu

Outline

Social bookmarking and information seeking

2008

Abstract

In recent years, there has been tremendous growth in shared bookmarking applications. Introduced in 2003, the del. icio. us social bookmark website was one of the first of this kind of application, and has enjoyed an early and large base of committed users. A flurry of similar offerings has since been unveiled [see (Hammond, et al., 2005) for a recent review].

FAQs

sparkles

AI

What are the limitations of current search engines for complex information tasks?add

The paper reveals that current search engines inadequately support users dealing with chronic illnesses or collaborative projects, limiting their effectiveness in complex scenarios (Marchionini & White, 2009). Instead, information seeking support systems (ISSS) should go beyond single-session lookups to assist users over time.

How do exploratory search systems enhance information-seeking experiences?add

Exploratory search systems leverage cumulative user interactions and adapt to evolving information needs, thus improving user engagement by allowing iterative exploration (Marchionini & White, 2009). These systems integrate social tagging and collaborative features to foster community-driven knowledge sharing.

What is the significance of user and task models in ISSS evaluation?add

User and task models are crucial for tailoring information retrieval systems to specific user contexts and goals; they establish benchmarks for assessing system effectiveness (Toms & O'Brien, 2008). This tailored approach ensures that diverse information-seeking behaviors are adequately supported.

How can longitudinal evaluation designs benefit information seeking support systems?add

Longitudinal evaluation designs track users' behaviors over time, thus providing insights into how information-seeking processes evolve, which enhances the adaptability of ISSS (Toms & O'Brien, 2008). This method contrasts with traditional evaluations focused on isolated search tasks.

What are the desired features for effective social bookmarking systems?add

Effective social bookmarking systems should facilitate easy resource tagging, content sharing, and collaboration, while ensuring accessibility across platforms to enhance community trust and knowledge sharing (Millen et al., 2007). Integrating these features directly into workplace tools can significantly improve co-worker interaction.

References (193)

  1. Anderson, J.R. Spanning seven orders of magnitude: A challenge for cognitive modeling. Cognitive Science, 26 (1) 2002, 85-112.
  2. Anderson, J.R. The adaptive character of thought. Lawrence Erlbaum Associates, Hillsdale, NJ, 1990.
  3. Bates, M.J. The design of browsing and berrypicking techniques for the online search interface. 13 1989, 407-424.
  4. Chi, E.H., Rosien, A., Suppattanasiri, G., Williams, A., Royer, C., Chow, C., Robles, E., Dalal, B., Chen, J. and Cousins, S. The Bloodhound Project: Automating discovery of Web usability issues using the InfoScent simulator. CHI 2003, ACM Conference on Human Factors in Computing Systems, CHI Letters, 5 (1) 2003, 505-512.
  5. Evans, B.M. and Chi, E.H. Towards a model of understanding social search Proceedings of the ACM 2008 conference on Computer supported cooperative work, ACM, San Diego, CA, USA, 2008.
  6. Fu, W. and Pirolli, P. SNIF-ACT: A model of user navigation on the World Wide Web. Human Computer Interaction, 22 (4) 2007, 355-412.
  7. Marchionini, G. Exploratory search: from finding to understanding. Communications of the ACM, 49 (4) 2006, 41-46.
  8. Marchionini, G. Information seeking in electronic environments. Cambridge University Press, Cambridge, England, 1995.
  9. Newell, A. and Card, S.K. The prospects for a psychological science in human-computer interactions. Human-Computer Interaction, 2 1985, 251-267.
  10. Pirolli, P. Information foraging: A theory of adaptive interaction with information. Oxford University Press, New York, 2007.
  11. Robertson, S.E. Theories and models in information retrieval. Journal of Documentation, 33 (2) 1977, 126-148.
  12. Wilkinson, D.M. Strong regularities in online peer production Proceedings of the 9th ACM conference on Electronic commerce, ACM, Chicago, Il, USA, 2008.
  13. REFERENCES
  14. Aula, A., Jhaveri, N., Käki, M. 2005. Information search and re-access strategies of experienced web users. In Proceedings of the 14th international conference on World Wide Web, 583-592.
  15. Kellar, M., Watters, C., & Shepherd, M. (2006). A Goal- based Classification of Web Information Tasks. In the Proceedings of the Annual Meeting of the American Society for Information Science and Technology (ASIS&T 2006), Austin, TX. (CD-ROM Proceedings)
  16. MacKay, B. & Watters, B. (2008) Exploring multi-session web tasks. Proceedings of CHI'2008, 1187-1196.
  17. Marchioni, G. (2006) Exploratory search: from finding to understanding. Communications of the ACM, 49, 4, 41-46.
  18. Newell, A. (1990). Unified theories of cognition. Cambridge, MA: Harvard University Press.
  19. Rose, D.E. & Levinson, D. (2004) Understanding user goals in Web search. Proceedings of the WWW 2004, 13-19.
  20. Rose, D. E. and Raju, S. (2007). Encouraging Exploration with Elroy: A New User Experience for Web Search. Proc. of ACM SIGCHI 2007 Workshop on Exploratory Search and HCI (ESI 2007), 52-55.
  21. White, R.W., Kules, B., Drucker, S.M., and Schraefel, M.C. (2006) Supporting exploratory search. Communications of the ACM, 49, 4, 37-39.
  22. REFERENCES
  23. Bates, M. (1990). Where should the person stop and the information search interface start. Information Processing and Management, 26(5), 575-591.
  24. Bates, M. J. (1979). Information search tactics. Journal of the American Society for Information Science, 30, 205-214.
  25. Bates, M. J. (1989). The design of browsing and berrypicking techniques for the on-line search interface. Online Review, 13(5), 407-431.
  26. Belkin, N. J. (1980). Anomalous states of knowledge as a basis for information retrieval. Canadian Journal of Information Science, 5, 133-143.
  27. Byström, K., & Hansen, P. (2002). Work tasks as units for analysis in information seeking and retrieval studies. In H. Bruce, R. Fidel, P. Ingwersen & P. Vakkari (Eds.), Emerging Frameworks and Methods (pp. 239-251). Greenwood Village, CO: Libraries Unlimited.
  28. Choo, C. W., Detlor, B., & Turnbull, D. (2000). Web Work: Information Seeking and Knowledge Work on the World Wide Web. Dordrecht, The Netherlands: Kluwer Academic Publishers.
  29. Dervin, B., & Nilan, M. (1986). Information needs and uses. In M. Williams (Ed.), Annual Review of Information Science and Technology (Vol. 21, pp. 3-33). White Plains, New York: Knowledge Industries.
  30. Ellis, D. (1989). A behavioral model for information retrieval system design. Journal of Information Science, 15(4-5), 237- 247.
  31. Fidel, R. (1985). Moves in online searching. Online Review, 9(1), 61-74.
  32. Furnas, G. W., & Rauch, S. J. (1998). Considerations for information environments and the NaviQue workspace. In Proceedings of the Third ACM Conference on Digital libraries, Pittsburgh, PA (pp. 79-88). New York: ACM Press.
  33. Ingwersen, P. & Järvelin, K. (2005). The Turn: Integration of Information Seeking and Retrieval in Context. Dordrecht, The Netherlands: Springer.
  34. Kari, J. (2005). Evolutionary information seeking: A case study of personal development and Internet searching. First Monday, 11(1). http://www.firstmonday.org/issues/issue11_1/kari/index.html
  35. Kules, B., Wilson, M., schraefel, m.c., Shneiderman, B. (in process) From Keyword Search to Exploration: How Result Visualization Aids Discovery on the Web.
  36. Marchionini, G. (1995). Information Seeking in Electronic Environments: Cambridge University Press.
  37. Pirolli, P. L., & Card, S. K. (1999). Information foraging. Psychological Review, 106(4), 643-675.
  38. Saracevic, T. (1996). Relevance reconsidered. In Information science: Integration in perspectives. Proceedings of the Second Conference on Conceptions of Library and Information Science (CoLIS 2). Copenhagen (Denmark), 14- 17 Oct.1996. pp. 201-218.
  39. L. Estabrook, E. Witt, and L. Rainie (2007). Information Searches That Solve Problems: How People Use the Internet, Libraries and Government Agencies When They Need Help. Pew/Internet. http://www.pewinternet.org/pdfs/Pew_UI_LibrariesR eport.pdf
  40. K. Martzoukou. (2004). A review of Web information seeking research: considerations of method and foci of interest. Information Research, 10(2) paper 215. http://InformationR.net/ir/10- 2/paper215.html]
  41. C.W. Choo, B. Detlor and D. Turnbull. (2000). Information seeking on the Web: an integrated model of browsing and searching. First Monday, 5 (2) http://firstmonday.org/issues/issue5_2/choo/index.ht ml
  42. E. Van Couvering (2006). Web Behaviour: Search Engines in Context. . http://personal.lse.ac.uk/vancouve/thesis/navigation.p df
  43. G. Marchionini (2006). Exploratory search: From finding to understanding. Communications of the ACM 49(4): 41-46.
  44. REFERENCES
  45. Bates, M. J. 1989. The Design of Browsing and Berrypicking Techniques for the Online Search Interface. Online Review, 13(5):407-24.
  46. Belkin, N. J., et al. 2001. Rutgers' TREC 2001 Interactive Track Experience. Proceedings of the Tenth Text Retrieval Conference (TREC-10).
  47. Belkin, N. J., Oddy, R. and Brooks, H. 1982. ASK for Information Retrieval. Journal of Documentation, 38, 61- 71 (part 1) & 145-164 (part 2).
  48. Dawkins, R. 1976. Memes: The New Replicators, in The Selfish Gene. Oxford University Press.
  49. Karlgren, J. and Franzen, K. 2000. Verbosity and Interface Design. SICS Technical Report T00:04, Swedish Institute for Computer Science, Stockholm.
  50. Rose, D. E. 2004. Why Is Web Search So Hard... to Evaluate? J. Web Eng. 3(3-4): 171-181.
  51. Rose, D. E. 2006. Reconciling Information-Seeking Behavior with Search User Interfaces for the Web. JASIST 57(6):797-799.
  52. Rose, D. E. and Stevens, C. 1997. V-Twin: A Lightweight Engine for Interactive Use, Proceedings of the Fifth Text Retrieval Conference (TREC-5).
  53. Rose, D. E. and Raju, S. 2007. Encouraging Exploration with Elroy: A New User Experience for Web Search. Proceedings of the ACM SIGCHI 2007 Workshop on Exploratory Search and HCI (ESI 2007), 52-55.
  54. Spink, A., Bateman, J., and Jansen, B. J. 1998. Users' Searching Behavior On The Excite Web Search Engine. WebNet 1998.
  55. Yahoo! Answers, http://answers.yahoo.com , accessed 12 June 2008.
  56. REFERENCES
  57. DeLone, W.H., & McLean, E.R. 2003. The DeLone and McLean Model of Information Systems Success: A Ten-Year Update, Journal of Management Information Systems 19(4 2003), 9-30.
  58. Freund, L, Toms, E.G., & Clarke, C. Modeling task-genre relationships for IR in the workplace. In Proceedings of the 28th Annual International ACM SIGIR Conference on Research and Development in Information Retrieval: SIGIR '05, Salvador, Brazil, August 15-18, 2005, pp. 441-448.
  59. Marchionini, G. Exploratory search: from finding to understanding. CACM 49(4 2006,) 41-46
  60. Taylor, R.S. Information use environments. Progress in Communication Sciences, 10 (1991), 217-255.
  61. Toms, E.G., Bartlett, J., Freund, L., Dufour, C., & Szigeti, S. Identifying the significant contextual factors of search. In: Proceedings of the SIGIR 2004 Workshop on Information Retrieval in Context, ,Sheffield (UK), 29 July2004.
  62. Toms, E.G., O'Brien, H., Kopak, R. & Freund, L (2005). Searching for relevance in the relevance of search. In Context: nature, impact, and role: COLIS5: Proceedings of the Fifth International Conference on Conceptions of Library and Information Sciences (Glasgow, UK, June 5-8, 2005), pp. 59-78. Lecture Notes in Computer Science 3507
  63. Ackerman, M. S.;McDonald, D. W. 1996. Answer Garden 2: merging organizational memory with collaborative help. In Proceedings of the 1996 ACM Conference on Computer Supported Cooperative Work (Boston, Massachusetts, United States, November 16 -20, 1996). M. S. Ackerman, Ed. CSCW '96. ACM, New York, NY, 97-105. DOI= http://doi.acm.org/10.1145/240080.240203
  64. Chi, E. H.; Mytkowicz, T. Understanding the efficiency of social tagging systems using information theory. Proceedings of the 19th ACM Conference on Hypertext and Hypermedia; 2008 June 19-21;
  65. Pittsburgh, PA. NY: ACM; 2008; 81-88.
  66. Evans, B.; Chi, E. H. Towards a Model of Understanding Social Search. In Proc. of Computer- Supported Cooperative Work (CSCW). ACM Press, 2008. San Diego, CA.
  67. B. M. Evans and S. K. Card. Augmented information assimilation: Social and algorithmic web aids for the information long tail. In Proc. CHI2008, ACM Press, pages 989-998, 2008.
  68. Kammerer, Y.; Nairn, R.; Pirolli, P. L.; Chi, E. H. Signpost from the Masses: Learning Effects in an Exploratory Social Tag Search Browser. (to appear) In Proc. CHI2009. Boston, MA.
  69. Marchionini, G. Exploratory search: From finding to understanding. Communications of the ACM, 49, 4 (2006), 41-46.
  70. White, R.W., Drucker, S.M., Marchionini, M., Hearst, M., schraefel, m.c. Exploratory search and HCI: designing and evaluating interfaces to support exploratory search interaction. Extended Abstracts CHI 2007, ACM Press (2007), 2877-2880.
  71. Briggs, P. and Smyth, B. 2007. Trusted search communities. In Proceedings of the 12th international Conference on intelligent User interfaces (Honolulu, Hawaii, USA, January 28 -31, 2007). IUI '07. ACM, New York, NY, 337-340.
  72. Chi, Ed. Social Search, this issue
  73. Golovchinsky, G., Pickens, J., and Back, M. A Taxonomy of Collaboration in Online Information Seeking. In Proceedings of the 1st International Workshop on Collaborative Information Retrieval. June 20, 2008. Available online at http://www.fxpal.com/?p=abstract&abstractID=454
  74. Linden, G., Smith, B. and York, J. Amazon.com Recommendations: Item-to-Item Collaborative Filtering. IEEE Internet Computing, 7 (1). 76-80.
  75. Millen, D. "Better Exploratory Search Through Social Bookmarking" this issue
  76. Morris, M. R. 2008. A survey of collaborative web search practices. In Proceeding of the Twenty-Sixth Annual SIGCHI Conference on Human Factors in Computing Systems (Florence, Italy, April 05 -10, 2008). CHI '08. ACM, New York, NY, 1657-1660.
  77. Morris, M. R. and Horvitz, E. 2007. SearchTogether: an interface for collaborative web search. In Proceedings of UIST '07. ACM, New York, NY, 3-12.
  78. Pickens, J., Golovchinsky, G., Shah, C., Qvarfordt, P., and Back, M. Algorithmic Mediation for Collaborative Exploratory Search. In Proceedings of SIGIR 2008, July 22-25, 2008, ACM Press.
  79. Shah, C. Toward Collaborative Information Seeking (CIS). In Proceedings of the 1st International Workshop on Collaborative Information Retrieval. June 20, 2008. Available online at http://workshops.fxpal.com/jcdl2008/submissions/tmpE1.pdf
  80. Smeaton, A.F., Lee, H., Foley, C., McGivney, S. and Gurrin, C., Físchlár-DiamondTouch: Collaborative Video Searching on a Table. In Proceedings of the SPIE Electronic Imaging-Multimedia Content Analysis, Management, and Retrieval (2006).
  81. NSF Workshop on Information Seeking Support Systems 50 http://ils.unc.edu/ISSS/
  82. REFERENCES
  83. Damianos, L. E., Cuomo, D., Griffith, J., Hirst, D. M. and Smallwood, J. (2007): 'Exploring the Adoption, Utility, and Social Influences of Social Bookmarking in a Corporate Environment'. In 40th Hawaii International Conference on System Sciences Waikoloa, HI, January, 2007, pp. 86-86.
  84. Dieberger, A. a. G., M. (2003): 'CoWeb -experiences with collaborative Web spaces.' In C. a. F. Lueg, D. : From Usenet to CoWebs: Interacting with Social Information Spaces. Springer-Verlag, New York, 2003, 155--166.
  85. Hammond, T., Hannay, T., Lund, B. and Scott, J. (2005): 'Social bookmarking tools (I) A General Review', D-Lib Magazine, 2005, ISSN: 1082-9873.
  86. Marchionini, G. (2006): 'Exploratory search: from finding to understanding', Communications of the ACM, 49, 4, 41-46.
  87. Millen, D. R., Yang, M., Whittaker, S. and Feinberg, J. (2007) Social bookmarking and exploratory search. ECSCW 2007, Limerick, Ireland, Sept 26-28, 2007.
  88. Millen, D. R., Feinberg, J. and Kerr, B. (2006): 'Dogear: Social bookmarking in the enterprise'. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems Montreal, Quebec, Canada, April, 2006, pp. 111-120.
  89. Munro, A. J., Hook, K., and Benyon (1999): Social Navigation of Information Space. Springer, London, 1999. NSF Workshop on Information Seeking Support Systems 54 http://ils.unc.edu/ISSS/ References
  90. Ahlberg, C., Williamson, C., and Shneiderman, B. (1992). Dynamic queries for information exploration: An implementation and evaluation. In Proc. ACM SIGCHI, pp. 619-626.
  91. Cool, C., Belkin, N.J., and Koenemann, J. (1996). On the potential utility of negative relevance feedback in interactive information retrieval. In Proc. ACM SIGIR, p. 341.
  92. Corbett, A.T., Koedinger, K.R., and Anderson, J.R. (1997). Intelligent tutoring systems. In Helander, M.G., Landauer, T.K., and Prabhu, P.V. (Eds.) Handbook of Human-Computer Interaction ( pp. 849-874). Amsterdam, The Netherlands: Elsevier Science.
  93. Foltz, P. and Landauer, T. (2007). Helping people find and learn from documents: Exploiting synergies between human and computer retrieval with SuperManual. In Landauer, T.K., McNamara, D.S., Dennis, S., and Kintsch, W. (Eds.), The Handbook of Latent Semantic Analysis, pp. 323-345. Mahwah, NJ: Erlbaum.
  94. Garfield, E. (1970). When is a negative search result positive? Essays of an Information Scientist, 1, 117- 118. Salton, G. and Buckley, C. (1990). Improving retrieval performance by relevance feedback. Journal of the American Society for Information Science, 41(4), 288-297.
  95. NSF Workshop on Information Seeking Support Systems 58 http://ils.unc.edu/ISSS/
  96. Belkin, N.J., Croft,W.B. Retrieval techniques, Annual review of information science and technology, vol. 22, Elsevier Science Inc., New York, NY, 1987.
  97. Bush, V. As We May Think. Monthly July 1945. http://www.theatlantic.com/doc/194507/bush. context histories for shared science.
  98. Dontcheva, M., Drucker, S.M., Salesin, D., Cohen, M.F., Relations, cards, and search templates: user-guided web data integration and layout. Proc. UIST '07. 2007: 61-70.
  99. Golovchinsky, Gene Queries? Links? Is there a difference? Proc. Of CHI, 1997, 407-414.
  100. Halasz, F.G., Moran, T.P., Trigg, R.H., Notecards in a nutshell, Proc of SIGCHI/GI, 1987: 45-52. NSF Workshop on Information Seeking Support Systems 68 http://ils.unc.edu/ISSS/
  101. Hong,L., Chi, E.C., Budiu, R., Pirolli, P., Nelson, L. SparTag.us: a low cost tagging system for foraging of web content. Proc AVI '08. 65-72.
  102. Huynh, D.F., Karger, D.R., Miller,R.C. Exhibit: lightweight structured data publishing. Proc WWW07: 737 -746.
  103. Marshall, C., Shipman, F.M. Spatial hypertext and the practice of information triage, Proc. Hypertext, 1997: 124- 133. Nelson, T. Literary Machines, Mindful Press, Sausalito, California, 1981. schraefel, m.c., Zhu,Y., Modjeska, D. Wigdor, D., Zhao, S. Hunter gatherer: interaction support for the creation and management of within-web-page collections. Proc. WWW '02: 172-181.
  104. White, R. W., Kules, B., Drucker, S. M. and schraefel, m. c. Introduction. Special Issue on Exploratory Search. CACM, 49 4 (2006). 36--39.
  105. Venolia, G. Backstory: A Search Tool for Software Developers Supporting Scalable Sensemaking. Microsoft Research. MSR-TR-2008-13.
  106. Wang, T.D., Plaisant, C., Quinn, A.J., Stanchak, R., Murphy, S. Shneiderman, B. Aligning temporal data by sentinel events: discovering patterns in electronic health records. Proc. Of CHI '08: 457-466. Bio: mc schraefel (lower case deliberate), PhD, Human Computer Interaction, Senior Research Fellow, Royal Academy of Engineering, Senior Lecturer, Electronics and Computer Science, University of Southampton, UK, mc+ieee@ecs.soton.
  107. REFERENCES
  108. Levi, M. D. (2008). Musings on Information Seeking Support Systems, ISSS 2008 position paper.
  109. Marchionini, G. (2006). Exploratory Search: From finding to understanding. Communication of ACM, 49(4): 41--46, 2006.
  110. Perrault, A. M. (2007). An exploratory study of biology teachers' online information seeking practices. School Library Media Research, 10. www.ala.org/ala/aasl/aaslpubsandjournals/slmrb/slmrcontents/vol ume10/biology.cfm Kules, B and Shneiderman, B. (2006). Improving the search experience by organizing Web search results with meaningful and stable categories. HCIL Symposium, University of Maryland, College Park, 2006
  111. White, R. W., Jose, J. M. and Ruthven, I. (2005). Using top- ranking sentences to facilitate effective information access. Journal of the American Society for Information Science and Technology, 56 (10), pp. 1113-1125.
  112. NSF Workshop on Information Seeking Support Systems 76 http://ils.unc.edu/ISSS/
  113. REFERENCES
  114. AHN, J.-W., BRUSILOVSKY, P., GRADY, J., HE, D., SYN, S.Y. Open user profiles for adaptive news systems: help or harm? in Proceedings of the 16th international conference on World Wide Web, WWW '07 (Banff, Canada, May 8-12, 2007), ACM 11-20.
  115. AHN, J.-W., BRUSILOVSKY, P., HE, D., GRADY, J., LI, Q. Personalized Web Exploration with Task Models in Proceedings of the 17th international conference on World Wide Web, WWW '08 (Beijing, China, April 21-25, 2008), ACM 1-10.
  116. HE, D., BRUSILOVSKY, P., GRADY, J., LI, Q., AHN, J.-W. How Up-to-date should it be? The Value of Instant Profiling and Adaptation in Information Filtering in Proceedings of the 2007 international conference on Web Intelligence, WI '07 (Silicon Valey, CA, USA, November 2-5, 2007), IEEE 699-705.
  117. JUNG, S., HARRIS, K., WEBSTER, J., HERLOCKER, J.L. SERF: Iintegrating human recommendations with search in Proceedings of ACM 13th Conference on Information and Knowledge Management, CIKM 2004 (Washington, DC, November 8-13, 2004 571-580.
  118. NSF Workshop on Information Seeking Support Systems 79 http://ils.unc.edu/ISSS/
  119. KANTOR, P.B., BOROS, E., MELAMED, B., MEÑKOV, V., SHAPIRA, B., NEU, Capturing human intelligence in the net, Communications of the ACM (2000) 43, 8, 112-116.
  120. MARCHIONINI, G. Exploratory search: From finding to understanding, Communications of the ACM (2006) 49, 4, 41-46.
  121. MICARELLI, A., GASPARETTI, F., SCIARRONE, F., GAUCH, S. Personalized search on World Wide Web, In: Brusilovsky, P., Kobsa, A., and Neidl, W., Eds. The Adaptive Web: Methods and Strategies of Web Personalization. Springer-Verlag, Berlin Heidelberg New York, 2007, 195-230.
  122. NSF Workshop on Information Seeking Support Systems 80 http://ils.unc.edu/ISSS/
  123. References
  124. Capra, R., and Marchionini, G. (2008). The Relation Browser Tool for Faceted Exploratory Search. Proceedings of the 2008 Conference on Digital Libraries (JCDL '08), Pittsburg, Pennsylvania, June 16 -20, 2008.
  125. Capra, R., Marchionini, G., Oh, J. S., Stutzman, F., and Zhang, Y. (2007). Effects of structure and interaction style on distinct search tasks. In Proceedings of the 2007 Conference on Digital Libraries (JCDL '07), Vancouver, BC, Canada, June 18 -23, 2007.
  126. Capra, R., and Marchionini, G. (2007). Visualizing Science and Engineering Indicators: Transitioning from Print to a Hybrid World. Technical Report TR-2007-07, School of Information and Library Science, University of North Carolina, Chapel Hill, NC.
  127. Marchionini, G. (2006) Exploratory search: From finding to understanding. CACM, 49(4), 41-46.
  128. Marchionini, G. and Brunk, B. (2003) Towards a general relation browser: A GUI for information architects. Journal of Digital Information, 4(1).
  129. Shneiderman, B. (1994). Dynamic Queries for Visual Information Seeking, IEEE Software, 11(6): 70-77.
  130. Zhang, J. and Marchionini, G. (2005). Evaluation and evolution of a browse and search interface: relation browser. In Proceedings of the 2005 National Conference on Digital Government Research (Atlanta, Georgia, May 15-18, 2005). ACM International Conference Proceeding Series, vol 89. Digital Government Research Center, 179-188.
  131. REFERENCES
  132. Hearst, M. (2006). Clustering versus faceted categorization for information exploration. Communications of the ACM 49, 4, 59-61.
  133. Marchionini, G. (2006). Exploratory search: From finding to understanding. Communications of the ACM 49, 4, 41-46.
  134. Prefuse Information Visualization Toolkit. www.prefuse.org
  135. Shneiderman, B. (2007). Creativity support tools: accelerating discovery and innovation. Communications of the ACM 50, 12, 20-32.
  136. SIMILE Timeline. http://simile.mit.edu/timeline/
  137. White, R.W., Kules, B., Drucker, S. and schraefel, m.c. (2006). Supporting exploratory search: Introduction. Communications of the ACM 49, 4, 36-39.
  138. Yahoo Design Pattern Library. http://developer.yahoo.com/ypatterns/ REFERENCES
  139. Aris, A. and Shneiderman, B., Designing semantic substrates for visual network exploration, Information Visualization Journal 6, 4 (2007), 1-20. doi: 10.1057/palgrave.ivs.9500162
  140. Aris, A. and Shneiderman, B., A node aggregation to reduce complexity in network visualizations using semantic substrates, University of Maryland Technical Report, Dept of Computer Science (February 2008).
  141. Card, S. K., MacKinlay, J. D., Shneiderman, B., Readings in Information Visualization: Using Vision to Think, Morgan Kaufmann Publishers, San Francisco, CA (1999).
  142. Chen, C., Top 10 Unsolved Information Visualization Problems, IEEE Computer Graphics & Applications (July/August 2005), 12-16.
  143. Kules, B. and Shneiderman, B., Users can change their web search tactics: Design principles for categorized overviews, Information Processing and Management (to appear 2008).
  144. North, C., Toward Measuring Visualization Insight., IEEE Computer Graphics & Applications 26, 3 (May- June 2006), 6-9
  145. Perer, A. and Shneiderman, B., Balancing systematic and flexible exploration of social networks, IEEE Symposium on Information Visualization and IEEE Transactions on Visualization and Computer Graphics 12, 5 (October 2006), 693-700.
  146. Perer, A. and Shneiderman, B., Systematic yet flexible discovery: Guiding domain experts during exploratory data analysis, Proc. ACM Conference on Intelligent User Interfaces (January 2008a).
  147. Perer, A. and Shneiderman, B., Integrating statistics and visualization: Case studies of gaining clarity during exploratory data analysis, ACM CHI2008 Conference, ACM, New York (April 2008b), 265-274.
  148. Saraiya, P., North, C., and Duca, K., An Insight-Based Methodology for Evaluating Bioinformatics Visualization, IEEE Trans. Visualization and Computer Graphics 11, 4 (July/Aug. 2005).
  149. Saraiya, P., North, C., and Duca, K., An Evaluation of Microarray Visualization Tools for Biological Insight", IEEE Symposium on Information Visualization 2004 (InfoVis 2004) (2004), 1-8.
  150. Shneiderman, B., The eyes have it: A task by data-type taxonomy for information visualizations, Proc. Visual Languages (Boulder, CO, Sept. 3-6). IEEE Computer Science Press, Los Alamitos, CA (1996), 336-343.
  151. Shneiderman, B. and Aris, A., Network visualization with semantic substrates, IEEE Symposium on Information Visualization and IEEE Transactions on Visualization and Computer Graphics 12, 5 (October 2006), 733-740.
  152. Shneiderman, B., Feldman, D., Rose, A., and Ferre, X. A., Visualizing digital library search results with categorical and hierarchical axes, Proc. 5 th ACM International Conference on Digital Libraries, ACM, New York (June 2000), 57-66.
  153. Thomas, J.J. and Cook, K.A. (eds.), Illuminating the Path: Research and Development Agenda for Visual Analytics, IEEE Press (2005).
  154. Alonso, O., Rose, D. E., & Stewart, B. (2008). Crowdsourcing for relevance evaluation. SIGIR Forum, 42(2), 10-16.
  155. Bates, M. J. (1989). The design of browsing and berrypicking techniques for the online search interface. Online Review, 13(5), 407-424.
  156. Cleverdon. C. W. (1997/1967). The Cranfield tests on index language devices. In K. Spark Jones & P. Willett (Eds.), Readings in Information Retrieval, San Francisco: Morgan Kaufman Publishers. (Reprinted from Aslib Proceedings, 173-192.)
  157. Dumais, S. T. & Belkin, N. J. (2005). The TREC Interactive Tracks: Putting the user into search. In E. M. Voorhees & D. K. Harman (Eds.) TREC: Experiment and Evaluation in Information Retrieval (pp. 123- 153), Cambridge, MA: MIT Press.
  158. Grimes, C., Tang, D. & Russell, D. (2007). Query logs alone are not enough. In Proceedings of WWW 2007 Workshop on Query Log Analysis: Social and Technological Changes. Järvelin, K., Price, S. L, Delcambre, L. M. L., & Nielsen, M. L. (2008). Discounted cumulated gain based evaluation of multiple-query IR sessions. Proceedings of the 30 th European Conference on Information Retrieval, Glasgow, Scotland.
  159. Kuhlthau, C. C. (1993). Seeking meaning: A process approach to library and information services. Norwood, NJ: Ablex.
  160. Ruthven, I. (2008). Interactive information retrieval. Annual Review of Information Science and Technology, 42, 43-91.
  161. Saracevic, T. (2007). Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: Nature and manifestations of relevance. Journal of the American Society for Information Science and Technology, 58(13), 1915-1933.
  162. Voorhees, E. M. & Harman, D. K. (2005). TREC: Experiment and Evaluation in Information Retrieval, Cambridge, MA: MIT Press. References:
  163. Borlund, P. (2003). The IIR evaluation model: A framework for evaluation of interactive information retrieval systems. Information Research, 8(3), paper no. 152. Retrieved September 19, 2008, from http://informationr.net/ir/8- 3/paper152.html.
  164. Kules, B., & Capra, R. (2008). Creating exploratory tasks for a faceted search interface. Paper presented at the Second Workshop on Human-Computer Interaction (HCIR 2008).
  165. White, R. W., Kules, B., Drucker, S. M., & schraefel, m.c. (2006). Supporting exploratory search. Communications of the ACM, 49(4), 36-39.
  166. NSF Workshop on Information Seeking Support Systems 106 http://ils.unc.edu/ISSS/ Making Sense of Search Result Pages [Position Paper] Jan O. Pedersen Yahoo! Inc. 2811 Mission College Blvd Santa Clara, CA 95054 jpederse@yahoo-inc.com
  167. REFERENCES
  168. Kalervo Järvelin and Jaana Kekäläinen, IR evaluation methods for retrieving highly relevant documents, SIGIR 2000, pages 41-48
  169. A.T. Duchowski, Eye Tracking Methodology: Theory and Practice, Springer-Verlag, New York, Inc., 2003
  170. Yun-Fang Juan and Chi-Chao Chang, An Analysis of Search Engine Switching Behavior using Click Streams, WWW 2005, Chiba Japan, pages 1050-1051
  171. NSF Workshop on Information Seeking Support Systems 108 http://ils.unc.edu/ISSS/
  172. O'Brien, H. (2008). Defining and Measuring Engagement in User Experiences with Technology. (Doctoral dissertation, Dalhousie University, 2008) (available from first author).
  173. Tague-Sutcliffe, J. (1992). Measuring the informativeness of a retrieval process. In Proceedings of the 15th Annual international ACM SIGIR Conference on Research and Development in information Retrieval (Copenhagen, Denmark, June 21 -24, 1992). N. Belkin, P. Ingwersen, and A. M. Pejtersen, Eds. SIGIR '92. ACM, New York, NY, 23-36.
  174. Toms, EG, O'Brien, H, Kopak, R & Freund, L. (2005). Searching for relevance in the relevance of search. In F. Crestani and I. Ruthven (eds.), Context: nature, impact, and role : COLIS5: Proceedings of the Fifth International Conference on Conceptions of Library and Information Sciences (Glasgow, UK, June 5-8), pp. 59-78. Lecture Notes in Computer Science 3507
  175. Elaine Toms is Canada Research Chair in Management Informatics and Associate Professor, Faculty of Management, Dalhousie University, Halifax, Nova Scotia, Canada. She received a PhD in library and information science from the University of Western Ontario. Contact her at etoms@dal.ca
  176. Heather O'Brien is an Assistant Professor at the School of Library, Archives and Information Studies, University of British Columbia, Vancouver, British Columbia, Canada. She received a PhD in Interdisciplinary Studies from Dalhousie University. Contact her at hlobrien@interchange.ubc.ca. NSF Workshop on Information Seeking Support Systems 110 http://ils.unc.edu/ISSS/
  177. Belkin, N. J., 2008. Some(What) Grand Challenges for Information Retrieval. ACM SIGIR Forum 42, 1 (June 2008), 47-54.
  178. Spärck Jones, K. 1988. A look back and a look forward. In: SIGIR '88. In Proceedings of the 11th Annual ACM SIGIR International Conference on Research and Development in Information Retrieval, 13-29.
  179. Saracevic, T. 1997. Users lost: reflections of the past, future and limits of information science. ACM SIGIR Forum 31, 2 (July 1997), 16-27.
  180. Ingwersen, P. and Järvelin, K. 2005. The turn. Integration of information seeking and retrieval in context. Springer.
  181. Voorhees, E. 2006. Building Test Collections for Adaptive Information Retrieval: What to Abstract for What cost? In First International Workshop on Adaptive Information Retrieval (AIR).
  182. Turpin, A. and Scholer, F. 2006. User performance versus precision measures for simple search tasks. In Proceedings of the 29th Annual international ACM SIGIR Conference on Research and Development in information Retrieval, 11-18.
  183. Saracevic, T. (2007). Relevance: A review of the literature and a framework for thinking on the notion in information science. Part II: nature and manifestations of relevance. Journal of the American Society for Information Science and Technology 58(3), 1915-1933.
  184. Cutting, D., Karger, D., Pedersen, J., and Tukey, J. 1992. Scatter/Gather: A Cluster-based Approach to Browsing Large Document Collections. In Proceedings of the 15th Annual ACM SIGIR International Conference on Research and Development in Information Retrieval, 318-329.
  185. Workshop on Faceted Search. 2006. In Proceedings of the 29th Annual ACM SIGIR International Conference on Research and Development in Information Retrieval.
  186. Von Ahn, L. 2006. Games with a Purpose. IEEE Computer 39, 6 (June 2006), 92-94.
  187. Von Ahn, L., Ginosar, S., Kedia, M., Liu, R., and Blum, M. 2006. Improving accessibility of the web with a computer game. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 79-82.
  188. NSF Workshop on Information Seeking Support Systems 113 http://ils.unc.edu/ISSS/
  189. Järvelin, K. and Kekäläinen, J. 2002. Cumulated gain-based evaluation of IR techniques. ACM Transactions on Information Systems 20, 4 (Oct. 2002), 422-446.
  190. Rao, R., Pedersen, J. O., Hearst, M. A., Mackinlay, J. D., Card, S. K., Masinter, L., Halvorsen, P., and Robertson, G. C. 1995. Rich interaction in the digital library. Communications of the ACM 38, 4 (Apr. 1995), 29-39.
  191. http://research.microsoft.com/ur/us/fundingopps/RFPs/Search_2006_RFP_Awards.aspx [ Google & IBM ] http://www.google.com/intl/en/press/pressrel/20071008_ibm_univ.html [ AOL ] http://query.nytimes.com/gs NSF Workshop on Information Seeking Support Systems 116 http://ils.unc.edu/ISSS/ Workshop Schedule Thursday June 26, 2008
  192. Workshop will be held in the Pleasants Family Assembly Room in Wilson Library on the UNC campus 8:00 Breakfast on-site 8:30 Introductions and overview 9:30 Breakout Session 1 (assigned based on project statements) 11:00 Reports back in plenary 12:30 Working lunch and Breakout Session 2 (lunch on-site) 2:00 Break 2:30 Plenary discussion: Key research themes 5:00 Close 6:00 Group dinner Carolina Inn Friday June 27, 2008
  193. Workshop will be held in the Pleasants Family Assembly Room in Wilson Library on the UNC campus 8:00 Breakfast on-site 8:30 Plenary discussion: Refine themes and map projects to themes 10:30 Break 11:00 Breakout Session 3. Aggregate and refine projects and plan writing 12:30 Working lunch (lunch on-site) 1:30 Plenary discussion of final report and task assignments 3:00 Wrap-up NSF Workshop on Information Seeking Support Systems 118 http://ils.unc.edu/ISSS/