Ubuntu Manpage: KiokuDB - Object Graph storage engine

NAME

       KiokuDB - Object Graph storage engine

VERSION

       version 0.57

SYNOPSIS

           use KiokuDB;

           # use a DSN
           my $d = KiokuDB->connect( $dsn, %args );

           # or manually instantiate a backend
           my $d = KiokuDB->new(
               backend => KiokuDB::Backend::Files->new(
                   dir        => "/tmp/foo",
                   serializer => "yaml",
               ),
           );

           # create a scope object
           my $s = $d->new_scope;

           # takes a snapshot of $some_object
           my $uuid = $d->store($some_object);

           # or with a custom ID:
           $d->store( $id => $some_object ); # $id can be any string

           # retrieve by ID
           my $some_object = $d->lookup($uuid);

           # some backends (like DBI) support simple searches
           $d->search({ name => "foo" });

           # others use GIN queries (DBI supports both)
           $d->search($gin_query);

DESCRIPTION

       KiokuDB is a Moose based frontend to various data stores, somewhere in between Tangram and Pixie.

       Its purpose is to provide persistence for "regular" objects with as little effort as possible, without
       sacrificing control over how persistence is actually done, especially for harder to serialize objects.

       KiokuDB is also non-invasive: it does not use ties, "AUTOLOAD", proxy objects, "sv_magic" or any other
       type of trickery.

       Many features important for proper Perl space semantics are supported, including shared data, circular
       structures, weak references, tied structures, etc.

       KiokuDB is meant to solve two related persistence problems:

       Transparent persistence
           Store  arbitrary  objects  without changing their class definitions or worrying about schema details,
           and without needing to conform to the limitations of a relational model.

       Interoperability
           Persisting arbitrary objects in a way  that  is  compatible  with  existing  data/code  (for  example
           interoperating with another app using CouchDB with JSPON semantics).

TUTORIAL

       If you're new to KiokuDB check out KiokuDB::Tutorial.

FUNDAMENTAL CONCEPTS

In order to use any persistence framework it is important to understand what it does and how it does it.

Systems like Tangram or DBIx::Class generally require explicit meta data and use a schema, which makes
them fairly predictable.

When using transparent systems like KiokuDB or Pixie it is more important to understand what's going on
behind the scenes in order to avoid surprises and limitations.

An architectural overview is available on the website: <http://www.iinteractive.com/kiokudb/arch.html>

The process is explained here and in the various component documentation in more detail.

Collapsing
When an object is stored using KiokuDB it's collapsed into an KiokuDB::Entry.

An entry is a simplified representation of the object, allowing the data to be saved in formats as simple
as JSON.

References to other objects are converted to symbolic references in the entry, so objects can be saved
independently of each other.

The entries are given to the KiokuDB::Backend for actual storage.

Collapsing is explained in detail in KiokuDB::Collapser. The way an entry is created varies with the
object's class.

Linking
When objects are loaded, entries are retrieved from the backend using their UIDs.

When a UID is already loaded (in the live object set of a KiokuDB instance, see KiokuDB::LiveObjects) the
live object is used. This way references to shared objects are shared in memory regardless of the order
the objects were stored or loaded.

This process is explained in detail in KiokuDB::Linker.

ROOT SET MEMBERSHIP

       Any object that is passed to "store" or "insert" directly is implicitly considered a member of  the  root
       set.

       This  flag implies that the object is an identified resource and should not be garbage collected with any
       of the proposed garbage collection schemes.

       The root flag may be modified explicitly:

           $kiokudb->set_root(@objects); # or unset_root

           $kiokudb->update(@objects);

       Lastly, root set membership may also be specified explicitly by the typemap.

       A root set member must be explicitly removed using "delete" or by removing it from  the  root  set.  Only
       non-members of the root set will be purged with any garbage collection scheme.

TRANSACTIONS

       On supporting backends the "txn_do" method will execute a block and commit the transaction at its end.

       Nesting  of  "txn_do"  blocks  is  always supported, though rolling back a nested transaction may produce
       different results on different backends.

       If the backend does not support transactions "txn_do" simply executes the code block normally.

CONCURRENCY

       Most transactional backends are also concurrent.

       KiokuDB::Backend::BDB and KiokuDB::Backend::CouchDB default to serializable transaction isolation and  do
       not  suffer  from  deadlocks, but serialization errors may occur, aborting the transaction (in which case
       the transaction should be tried again).

       KiokuDB::Backend::Files provides good concurrency support but will only  detect  deadlocks  on  platforms
       which  return "EDEADLK" from "flock".  Directory::Transactional may provide alternative mechanisms in the
       future.

       Concurrency support in KiokuDB::Backend::DBI depends on the database.  SQLite  defaults  to  serializable
       transaction isolation out of the box, whereas MySQL and PostgreSQL default to read committed.

       Depending  on your application read committed isolation may be sufficient, but due to the graph structure
       nature of the data repeatable reads or serializable level isolation is highly recommended. Read committed
       isolation generally works well when each row in the database is more or less independent of  others,  and
       various constraints ensure integrity. Unfortunately this is not the case with the graph layout.

       To  enable  stronger  isolation  guarantees  see "Transactions" in KiokuDB::Backend::DBI for per-database
       pointers.

ATTRIBUTES

       KiokuDB uses a number of delegates which do the actual work.

       Of these only "backend" is required, the rest have default definitions.

       Additional attributes that are not commonly used are listed in "INTERNAL ATTRIBUTES".

       backend
           This attribute is required.

           This must be an object that does KiokuDB::Backend.

           The backend handles storage and retrieval of entries.

       typemap
           This is an instance KiokuDB::TypeMap.

           The typemap  contains  entries  which  control  how  KiokuDB::Collapser  and  KiokuDB::Linker  handle
           different types of objects.

       allow_classes
           An array references of extra classes to allow.

           Objects blessed into these classes will be collapsed using KiokuDB::TypeMap::Entry::Naive.

       allow_bases
           An array references of extra base classes to allow.

           Objects derived from these classes will be collapsed using KiokuDB::TypeMap::Entry::Naive.

       allow_class_builders
           If true adds KiokuDB::TypeMap::ClassBuilders to the merged typemap.

           It's    possible    to    provide   a   hash   reference   of   options   to   give   to   "new"   in
           KiokuDB::TypeMap::ClassBuilders.

       check_class_versions
           Controls whether or not the class versions of objects are checked on load.

           Defaults to true.

       class_version_table
           A table of classes and versions that is passed to the  default  typemap  entry  for  Moose/Class::MOP
           objects.

           When  a  class version has changed between the time that an object was stored and the time it's being
           retrieved, the data must be converted.

           See KiokuDB::TypeMap::Entry::MOP for more details.

METHODS

connect $dsn, %args
DWIM wrapper for "new".

$dsn represents some sort of backend (much like DBI dsns map to DBDs).

An example DSN is:

my $dir = KiokuDB->connect("bdb:dir=path/to/data/");

The backend moniker name is extracted by splitting on the colon. The rest of the string is passed to
"new_from_dsn", which is documented in more detail in KiokuDB::Backend.

Typically DSN arguments are separated by ";", with "=" separating keys and values. Arguments with no
value are assumed to denote boolean truth (e.g. "jspon:dir=foo;pretty" means "dir => "foo", pretty
=> 1"). However, a backend may override the default parsing, so this is not guaranteed.

Extra arguments are passed both to the backend constructor, and the "KiokuDB" constructor.

Note that if you need a typemap you still need to pass it in:

KiokuDB->connect( $dsn, typemap => $typemap );

The DSN can also be a valid JSON string taking one of the following forms:

dsn => '["dbi:SQLite:foo",{"schema":"MyApp::DB"}]'

dsn => '{"dsn":"dbi:SQLite:foo","schema":"MyApp::DB"}'

This allows more complicated arguments to be specified accurately, or arbitrary options to be
specified when the backend has nonstandard DSN parsing (for instance KiokuDB::Backend::DBI simply
passes the string to DBI, so this is necessary in order to specify options on the command line).

configure $config_file, %args
TODO

new %args
Creates a new directory object.

See "ATTRIBUTES"

new_scope
Creates a new object scope. Handled by "live_objects".

The object scope artificially bumps up the reference count of objects to ensure that they live at
least as long as the scope does.

This ensures that weak references aren't deleted prematurely, and the object graph doesn't get
corrupted without needing to create circular structures and cleaning up leaks manually.

lookup @ids
Fetches the objects for the specified IDs from the live object set or from storage.

store @objects
store %objects
store_nonroot @objects
store_nonroot %objects
Recursively collapses @objects and inserts or updates the entries.

This performs a full update of every reachable object from @objects, snapshotting everything.

Strings found in the object list are assumed to be IDs for the following objects.

The "nonroot" variant will not mark the objects as members of the root set (therefore they will be
subject to garbage collection).

update @objects
Performs a shallow update of @objects (referents are not updated).

It is an error to update an object not in the database.

deep_update @objects
Update @objects and all of the objects they reference. All references objects must already be in the
database.

insert @objects
insert %objects
insert_nonroot @objects
insert_nonroot %objects
Inserts objects to the database.

It is an error to insert objects that are already in the database, all elements of @objects must be
new, but their referents don't have to be.

@objects will be collapsed recursively, but the collapsing stops at known objects, which will not be
updated.

The "nonroot" variant will not mark the objects as members of the root set (therefore they will be
subject to garbage collection).

delete @objects_or_ids
Deletes the specified objects from the store.

Note that this can cause lookup errors if the object you are deleting is referred to by another
object, because that link will be broken.

set_root @objects
unset_root @objects
Modify the "root" flag on the associated entries.

"update" must be called for the change to take effect.

txn_do $code, %args
txn_do %args
scoped_txn $code
Executes $code within the scope of a transaction.

This requires that the backend supports transactions (KiokuDB::Backend::Role::TXN).

If the backend does not support transactions, the code block will simply be invoked.

Transactions may be nested.

If the "scope" argument is true an implicit call to "new_scope" will be made, keeping the scope for
the duration of the transaction.

The return value is propagated from the code block, with handling of list/scalar/void context.

"scoped_txn" is like "txn_do" but sets "scope" to true.

txn_begin
txn_commit
txn_rollback
These methods simply call the corresponding methods on the backend.

Like "txn_do" these methods are no-ops if the backend does not support transactions.

search \%proto
search @args
Searching requires a backend that supports querying.

The "\%proto" form is currently unspecified but in the future should provide a simple but consistent
way of looking up objects by attributes.

The second form is backend specific querying, for instance Search::GIN::Query objects passed to
KiokuDB::Backend::BDB::GIN or the generic GIN backend wrapper KiokuDB::GIN.

Returns a Data::Stream::Bulk of the results.

root_set
Returns a Data::Stream::Bulk of all the root objects in the database.

all_objects
Returns a Data::Stream::Bulk of all the objects in the database.

grep $filter
Returns a Data::Stream::Bulk of the objects in "root_set" filtered by $filter.

scan $callback
Iterates the root set calling $callback for each object.

object_to_id
objects_to_ids
id_to_object
ids_to_objects
Delegates to KiokuDB::LiveObjects

directory
Returns $self.

This is used when setting up KiokuDB::Role::API delegation chains. Calling "directory" on any level
of delegator will always return the real KiokuDB instance no matter how deep.

GLOBALS

       $SERIAL_IDS
           If set at compile time, the default UUID generation role will use serial IDs, instead of UUIDs.

           This  is useful for testing, since the same IDs will be issued each run, but is utterly broken in the
           face of concurrency.

INTERNAL ATTRIBUTES

       These attributes are documented for completeness and should typically not be needed.

       collapser
           KiokuDB::Collapser

           The collapser prepares objects for storage,  by  creating  KiokuDB::Entry  objects  to  pass  to  the
           backend.

       linker
           KiokuDB::Linker

           The linker links entries into functioning instances, loading necessary dependencies from the backend.

       live_objects
           KiokuDB::LiveObjects

           The live object set keeps track of objects and entries for the linker and the resolver.

           It  also  creates scope objects that help ensure objects don't garbage collect too early ("new_scope"
           in KiokuDB::LiveObjects, KiokuDB::LiveObjects::Scope), and transaction scope objects used by "txn_do"
           (KiokuDB::LiveObjects::TXNScope).

       typemap_resolver
           An instance of KiokuDB::TypeMap::Resolver. Handles actual lookup and compilation of typemap  entries,
           using the user typemap.

VERSION CONTROL

       KiokuDB    is    maintained   using   Git.   Information   about   the   repository   is   available   on
       <http://www.iinteractive.com/kiokudb/>

AUTHOR

       Yuval Kogman <nothingmuch@woobling.org>

COPYRIGHT AND LICENSE

       This software is copyright (c) 2014 by Yuval Kogman, Infinity Interactive.

       This is free software; you can redistribute it and/or modify it under  the  same  terms  as  the  Perl  5
       programming language system itself.

perl v5.34.0                                       2022-05-23                                       KiokuDB(3pm)