Loading Large Tables

Fast and Exo Loading %. More...

Detailed Description

Fast and Exo Loading %.

---

Class Documentation

load_mega_clause/1

class load_mega_clause/1

load_mega_clause( +Stream )

properties: deterministic

ail Load a single predicare composed of facts with the same size

load_db/1

class load_db/1

load_db(+ Files)

load_db( +Files )

Load a database of ground facts All facts must take up the same amount of storage, so that a fact $I$ can be accessed at position P[I-1] This representation thus stores the facts as a huge continuous array, the so-called mega clause

See [for] a motivation for this technique YAP implements this optimization by default whenever it loads a large number of facts (see Yap_BuildMegaClause(PredEntry *ap) ) for the details On the other hand, loading the data-base will cause fragmentation because individual facts facts need extra headers ands tails, and because often new atoms will be stored in the Symbol Table, see LookupAtom(const char *atom) The main advantage of load_db/1 is that it allocates the necessary memory only once Just doing this may halve total memory usage in large in-memory database-oriented applications

Note

Implementation

YAP implements load_db/1 as a two-step non-optimised process First, it counts the nmuber of facts and checks their size Second, it allocates and fills the memory The first step of the algorithm is implemented by dbload_get_space(), and the second by dbload_add_facts()

db_files/1 itself is just a call to load_files/2

properties: deterministic

Load files each one containing as single predicare composed of facts with the same size