The constantly evolving Web reects the evolution of society in the cyberspace.
Projects like the Open Directory Project (dmoz.org) can be understood as a
collective memory of society on the Web. The main assumption is that such
collective Web memories evolve when a certain cognition level about a concept
has been exceeded. In the scope of our work we analyse the New York Times
archive for concept detection. There are several approaches to the concept
modelling. We introduce an alternative model for concepts, which does not make
any additional assumptions about types of contained entities or the number of
entities in the corpus. Moreover, the proposed distributed concept computation
algorithm enables the large scale archive analysis. We also introduce a model
of cognition level and explain how it can be employed to predict changes in the
category system of DMOZ.