Comments on: Indexing Dynamic Databased Content

By: soulemane

soulemane — Tue, 04 Jan 2011 18:29:58 +0000

Hi !

I read with a lot on enthusiasm your article as well as comments.
In fact i’m a student working on a project regarding
How to index dynamic documents,what kind of algorithm to use for indexing this type of documents .
Your suggestion is highly welcome
thanks

By: Dynamic URls Vs Static Content – Google Indexes Best at Search Engine Optimization Tips, SEO for Blogs, Search Tips

Wed, 14 Jan 2009 05:00:28 +0000

[...] In case you’re a large site, I recommend reading Arnold’s Indexing dynamic content [...]

By: Stephen E. Arnold

Stephen E. Arnold — Sat, 02 Aug 2008 20:08:13 +0000

Nitan,

Thank you for writing. There may be “tension” between the Web site and the indexing system. The Web site has one or a limited set of data. The Web site sees those data as having significant value. The indexing system sees the Web site as farmer with corn, essentially data are data. The indexing system, therefore, creates value by harvesting the Web site and creating a meta-construct that has higher fungible value than the single Web site’s data. Therefore, we have tension. Lots of tension.

Stephen Arnold, August 2, 2008

By: Nitin Anand

Nitin Anand — Sat, 02 Aug 2008 17:17:05 +0000

Stephen, great article.
Just one thought – will it not be better if we let the websites take more responsibility of what search leads to which direction.
I feel the websites knows more than the search engines in terms of what information is stored, be it static or dynamic, within their system. so why not have the user go to the website to avail of the special promotion instead of first going to a search engine.
your comments…

thanks
Nitiin

By: Stephen E. Arnold

Stephen E. Arnold — Thu, 05 Jun 2008 22:14:04 +0000

Watch for an interview with one of the “deep Web” gurus, Abe Lederman. His interview for the Search Wizards Speak series on the ArnoldIT.com Web site will run either Monday or Tuesday, June 9 or 10. Mr. Lederman is preparing for a trip, but he assures me that I will have some final inputs before he wallows in the luxury of a 20 hour flight on one of the Trailways of the air.

Stephen Arnold, June 5, 2008

By: The Deep Web « Blogs will be Blogs…

The Deep Web « Blogs will be Blogs… — Thu, 05 Jun 2008 21:18:24 +0000

[...] is now working on indexing dynamic web page content and therefore poking into the deep web. This article here talks about that and mentions some of the players in deep web searching such as [...]

By: Stephen E. Arnold

Stephen E. Arnold — Fri, 02 May 2008 13:43:21 +0000

I appreciate your post. In these Web log essays, I often try to make one point. I certainly agree with your comments. In the back of my mind are the Guha patent applications. These contain provisions for discovering fields and missing data if a Web site does not provide programmatic instructions. Southwest is an interesting case. Under the airline’s present management, I’m not seeing SWA flights in the aggregation services I use.
Stephen Arnold, May 2, 2008

By: Jeff

Jeff — Thu, 01 May 2008 18:08:05 +0000

You didn’t mention that the content managers, ie entities hosting deep web data, can create dynamic site maps based off of the dynamic content, which allows the enterprise crawlers easy access to the deep web content. Fairly trivial to do so and very effective in getting this data into the enterprise search engine indexes.

Good article, however I disagree with the blanket statement “Second, the people operating dynamic sites want to protect their data from aggregators.” Not necessarily true. Take for instance your query of ‘flights from Louisville, Kentucky, to Baltimore, Maryland’. I would think that any SouthWest Airlines shareholder would be quite happy if I typed this into Google and got the SouthWest page of flights, with a handy link to purchase tickets. Your statement is probably true in some cases, therefore the question to answer is “What is the main objection to the public finding data available in publicly available systems (given that the front page to the web is Google, MSN, Yahoo; and does this objection describe a majority of deep web content owners?”

Content for another article?

By: E-researching» ?????? » SearchCap: The Day In Search, April 22, 2008

E-researching» ?????? » SearchCap: The Day In Search, April 22, 2008 — Tue, 22 Apr 2008 22:41:16 +0000

[...] Indexing Dynamic Databased Content : Beyond Search, arnoldit.com [...]

By: Stephen E. Arnold

Stephen E. Arnold — Mon, 21 Apr 2008 15:09:45 +0000

Thanks for writing. I would like to learn more and maybe put up a short profile of what you and your team are doing. I don’t include an exhaustive list of vendors in these Web log posts. In the first three editions of Enterprise Search Report I profiled about 32 vendors. In Beyond Search, my most recent study, I profiled an additional 24 vendors. I have a list of more than 300 companies in this space, and I would be delighted to add your firm to the list.
Stephen Arnold, April 21, 2008, 11 09 am