Enterprise Search: Evidence It Is a Commodity

January 17, 2015

I was browsing through some information gathered by Overflight last week. I cam across an interesting page showing Libraries Australia Architecture Overview. Here’s a miniature of the diagram. The link provides a larger version. Where is search? Well, it is in the middle, represented by a purple storage icon.


The search system is Solr. I find this interesting for several reasons:

First, Solr replaced the Australian-developed TeraText search system, which I think is pretty good. TeraText was a commercial product, and Solr is an open source system.

Second, Solr is a component in a far larger system. No surprise here, but the diagram makes clear that search is a utility supporting many other library functions. For vendors who make search the fabric for a large-scale application, the Libraries Australia team may want you to give them a lecture about ways to improve their system.

Third, Libraries Australia has a number of systems, each of which presumably has its native search tools. The implication is that Solr provides one screen access to these diverse resources. I wonder if the Oracle DBA uses Solr instead of the native Oracle tools. My thought is that the Solr champions see no reason to fool with Oracle command lines. The DBA, on the other hand, may see information access from a different point of view.

Net net: A commercial account closes, and an open source account begins. Does this fact suggest that closing deals for proprietary search systems might be more difficult in 2015?

Stephen E Arnold, January 17, 2015

EU Decision on Digitization by Library

October 8, 2014

The European Court of Justice recently issued an interesting ruling. Intellectual Property Watch reports that “Libraries May Be Permitted to Digitise Books Without Copyright Owner’s Consent, EU High Court Rules.” The decision says libraries may digitize works to make them available at electronic reading stations, but draws the line at printing them out or copying them to a USB drive. The precipitating event seems to have occurred in Germany, where a university library refused to purchase an e-book and, instead, chose to place the book in its computer system without the publisher’s consent.

The EU copyright directive issued in 2001 does carve out an exception for libraries to make content available electronically. Unsure whether the above usage is covered by the exception, the German Federal Court of Justice consulted the EU court for clarification. Writer Dugie Standeford explains:

“The ECJ held that even if a rights owner offers a library a licence agreement for use of the work on appropriate terms, the library may take advantage of the exception, since it otherwise could not fulfil its core mission of promoting research and private study. The directive doesn’t bar governments from giving libraries the right to digitise books, and, if necessary, from making the material available on dedicated computers, the court said.

“But the right of communication which public libraries may hold doesn’t allow people to print out the works on paper or store them on USB sticks, because those are acts of reproduction which aim to create a new copy of the digital copy, the court said. Nevertheless, it added, member states may provide an exception or limitation that allows library users to print the works or store them on a USB stick, so long as compensation is paid to the rightsholder.”

Concerns have been raised about how compensation might be collected for such copies. On the other hand, the decision has been hailed as a win for libraries and archives. The question of online access is not specifically mentioned in the ruling, but it does limit the exception to “dedicated terminals.” As remote access to information becomes more and more standard, we may have another clarification to look forward to.

Cynthia Murrell, October 08, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

Good-Bye Court Documents

September 22, 2014

The Internet makes it easier to access information, including documents from the government. While accessing government documents might cost a few cents, it is amazing that the information can be accessed within a few mouse clicks. BoingBoing, run by the infamous Cory Doctorow, notes that five important US courts are removing their documents from the Internet in “As Office Of US Courts Withdraws Records For Five Top Benches, Can We Make Them Open?”

The court documents are housed on the PACER system, most notable for charging users ten cents a page to access information. Doctorow advocates for free information and stopping governments from spying on its citizens. It is not surprising that he supports reopening these documents, along with the Free Law Project, Internet Archive, and Public.Resource.Org.

The plea reads:

“Our judiciary is based on the idea that we conduct justice public, not in star chambers and smoke-filled back rooms. Our system of justice is based on access to the workings of our courts, and when you hide those workings behind a pay wall, you have imposed a poll tax on access to justice. Aaron [Swartz] and many others believed very deeply in this principle and we will continue to fight for access to justice, equal protection, and due process. These are not radical ideas and the Administrative Office of the U.S. Courts should join us in our commitment.”

Swartz is known for working against Internet censorship bills, so joining Doctorow and the others will get the right backers to make these documents available again. You can fight city hall and win, especially if you are a technology enthusiast with legal aid.

Whitney Grace, September 22, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Cross Books Off the Back To School List

September 19, 2014

No more pencils, no more books, no more…wait! No more books? According to an io9 article, “The First College In The US To Open Without Any Books In Its Library” dead tree items might be a thing of the past at least for one university. Florida Polytechnic University in Lakeland recently opened with 550 students as part of its first class. The brand new campus has the usual campus buildings, including a library. The library, though, is different from your typical archive of knowledge: it is the nations first all digital library collection.

All of the books in the library are available via software that allows the students to download ebooks and what we can assume access to academic databases. An even bigger change is that librarians will not man the reference desk, because its name has been switched to the “success desk.” Librarians will instead be train students on information literacy and how to access electronic resources. Students will still be able to access books via interlibrary loan from other universities. They will also be able to decide how Florida Polytechnic spends its $60,000 library budget.

These are some good ideas in theory, but the technology is not up to being a free and browseable collection:

“Defenders of brick-and-mortar bookstores have argued the opposite, saying that the experience of wandering among bookshelves inspires serendipitous discoveries, while searching a database yields only the exact results you set out to find. While you can find related books in a database, it is unlikely you’ll stumble across an unrelated but helpful book while searching for another one by title.”

In most cases, students are also limited to how many times the can download and read an ebook. Digital licenses can track that kind of usage, so how long will some of these ideas last?

Whitney Grace, September 19, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

The New Digital Library

July 1, 2014

Libraries were revolutionized when online public access catalogs did away with the old card catalog and made the entire collection available on the Internet. Libraries across the world soon got the idea to “dump” their catalogs in one big database. Worldcat.org was born and people can locate any book in any library that shares their catalog. Next came ebooks where people could download any book to a digital device for a fee and libraries caught on to with too. Enter Netflix, Hulu, Amazon Prime, and other streaming services and someone got the bright idea to apply the same business model to books.

Public libraries may be lamenting, but people who don’t mind paying a low nominal fee will enjoy unlimited books a month. The newest member to the downloadable entertainment service is Scribd:

“Scribd is your personal digital library, where you have unlimited access to the world’s largest collection of e-books and written works. Our premium subscription service offers over 400,000 books from over 900 publishers, including New York Times bestsellers, literary classics, groundbreaking non-fiction, and reader favorites in every genre. We also have millions of user-uploaded written works, from landmark court filings to academic papers from scholars around the world.”

Is this a good thing? Yes. It creates more competition for Amazon and gives readers more options to access books. Does it mean libraries will go away? No, people still want things for free. This is a great move for publishers and takes advantage of new platforms. Though there is nothing that can replace a good paperback.

Whitney Grace, July 01, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Libraries Hurt If Net Neutrality Dies

June 12, 2014

The Washington Post blog The Switch interviewed the American Library Association’s Director of Government Relations Lynne Bradley in the article “Why The Death of Net Neutrality Would Be A Disaster For Libraries” about how libraries would be adversely affected without net neutrality. As public institutions with most of their resources online, libraries rely on free Web to serve their users and provide information. With budgets already stretched to their limits, libraries will not be able to afford to pay to ISPs. More people are going to the public libraries to have access to the Internet and other digital services. If that is taken away, not only will libraries suffer, but public education institutions will also suffer.

“In a way, not having a truly open Internet is like privatizing all of the Internet. Our nation was built on the concept of public schools, and public libraries are part of that, even the universal service fund at the FCC. These are part of our nation’s public policies that say as all educated, as all can have public libraries, as all can have public phone service, it’s best for the country as a whole.”

Libraries are at the forefront of how people consume information. Even if most of the resources are digital, the users represent a large percentage of people who go unnoticed when it comes to usage. While it’s safe to assume that everyone has Internet access, it needs to be taken into account how they connect. Raise the price of Internet access and it will harm those who need it most.

Whitney Grace, June 12, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Envisioning Libraries of the Future

May 27, 2014

The debate over the future of libraries continues to rage. Will they even continue to exist? If they do, they will surely not continue on unchanged. This librarian wants to weigh in, and The Slate story, “What Will Become of the Library?” serves as a lovely backdrop for discussion.

First of all, yes, libraries will have to change. But libraries are poised to meet an increasingly large gap left by the digital divide (those who have internet access and those who don’t):

“Eventually, the Venn diagram of those who lack smartphones and those who lack homes may nearly overlap exactly. Libraries are well positioned to serve many of the needs of this demographic, the dispossessed of the digital age.

Now, this is assuming that all things remain constant, and we know they definitely will not. There is no way to truly know how this will unravel. The decline of books is inevitable, but will proceed much more slowly than most folks predict. Most importantly, libraries will continue to exist if the community needs them to.

The article concludes:

Libraries will only survive if the communities they serve want and need them to. It would be a tragedy of historic proportions if, for instance, the public library system that Carnegie endowed and inspired is dismantled in the coming decades, but it’s a real possibility. In the end, it’s up to us—scholars, makers, and artists, seekers of community, access, and safe haven, and above all, readers in the old, human sense of the word—to rise to the level of these monuments we’ve built.”

Well said. This librarian is optimistic.

Emily Rae Aldridge, May 27, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

The Digital Public Library Turns One and Celebrates with Big Partnerships

May 7, 2014

It is interesting that no one thought of creating a public library entirely online until last year. There are many libraries and archives that have online components to their collections, but nothing matches the sheer volume found on the Digital Public Library of America. Arstechnica reports that “Digital Public Library Of America To Add Millions Of Records To Its Archive” with six partnerships, including the California Digital Library, the Montana Memory Project, Indiana Memory Project, the US Government Printing Office, Connecticut Digital Archive, and the J. Paul Getty Trust. Also the New York Public Library will increase its catalog items in the DPLA to over one million.

The DPLA is a platform that connects digital archives and libraries through one hub. DPLA allows its users to search through all its partners’ databases through one portal, think of it as Google sans advertisements and Web site results.

“In its first year, one of the DPLA’s most important goals has been to catalog and connect to as many digital works as possible. Ars caught up with [DPLA Executive Director Dan] Cohen again this year, and he told us that the organization has made it a priority to help public libraries digitize their works using a grant from the Bill and Melinda Gates Foundation to advocate for putting resources online. “We are helping to train public librarians with the digital skills they will need for the twenty-first century and to participate in a large-scale digital project like DPLA,” Cohen wrote in an e-mail.

The DPLA also has several third party apps that help users take advantage of its immense wealth of knowledge. The best thing about the DPLA is that access to its items are entirely free! Researchers don’t have to worry about subscribing to a database or paying for to locate records. This is how research should be done everywhere!

Whitney Grace, May 07, 2014
Sponsored by ArnoldIT.com, developer of Augmentext

Pope Francis Embraces the Internet in Vatican Library Archives Digitizing Scheme

April 4, 2014

The gloriously irreverent article on Gawker titled Japanese Tech Company to Digitize Vatican Library Archives provides insight into the project undertaken by “the chilliest pope” Pope Francis of digitizing the Vatican library archives. The archives extensive collection of documents will cost over 20 million dollars to transfer, and the work has already been started by Japanese tech company NTT Data. The article mentions that this is not the first instance of Japanese company aiding the Vatican. There is plenty of work to keep NTT Data busy. The article explains,

“The goal is to make all 82,000 of the Vatican library’s manuscripts available for browsing from the darkest corners of our digital world without ever having to board a plane. Some of the first documents to become available include “copies of works of classical Greek and Latin literature and mediaeval and Renaissance illuminated manuscripts.” Get your browsing fingers ready. Excitement abounds.”

Pope Francis has proved himself a more progressive pope by many of his actions, and this is a similar embrace of current cultural trends. Making the Vatican’s archives available to the general public sounds like a very democratic vision, contrasted with the stereotypical secretive atmosphere the Vatican is usually associated with. But is it all really progress? Perhaps that depends on the man himself.

Chelsea Kerwin, April 04, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

A Wealth of Words at Open Library

March 28, 2014

Looking for a book, any book? Head on over to Open Library, whose laudable goal is to provide “one web page for every book.” Though they have not reached that possibly unachievable aim, the many contributors have made a dent; the page hosts over 20 million documents. Open Library’s home page explains that, much like Wikipedia, the whole project is “open”—open software, open data, open documentation, and an invitation for anyone to “fix a typo, add a book, or write a widget.” The site’s About page explains:

“At its heart, Open Library is a catalog. The project began in November 2007 and has been inhaling catalog records from some of the biggest libraries in the world ever since. We have well over 20 million edition records online, provide access to 1.7 million scanned versions of books, and link to external sources like WorldCat and Amazon when we can. The secondary goal is to get you as close to the actual document you’re looking for as we can, whether that is a scanned version courtesy of the Internet Archive, or a link to Powell’s where you can purchase your own copy.”

How is this different from Project Gutenberg? Well, while Open Library strives to provide at the very least a link to every book in existence, Gutenberg seems content to host a mere 30,000 or so volumes. Part of what makes Open Library’s approach possible is the “borrow modern ebooks” option—users check out and relinquish access to digital copies of works still in copyright. The site pursues the rights to lend these books several ways. Some are contributed by the more-than-1,000 libraries participating in the In-Library Lending Program. There is also a smaller collection of about 11,000 ebooks contributed by the Internet Archive. Open Library also links to WorldCat and OverDrive.com to help users find any works not available in their expanding collection. Bookmark this one for the next time you need to find a book. Any book.

Cynthia Murrell, March 28, 2014

Sponsored by ArnoldIT.com, developer of Augmentext

Next Page »