Berkeley DB is a C library which provides a consistent interface to a number of database formats. DB_File provides an interface to all three of the database types currently supported by Berkeley DB.
The file types are:
A default hashing algorithm, which will be adequate for most applications, is built into Berkeley DB. If you do need to use your own hashing algorithm it is possible to write your own in Perl and have DB_File use it instead.
As with the DB_HASH format, it is possible to provide a user defined Perl routine to perform the comparison of keys. By default, though, the keys are stored in lexical order.
In addition to the tie() interface, it is also possible to use most of the functions provided in the Berkeley DB API.
The parameter type
is an enumeration which specifies which of the 3
interface methods (DB_HASH, DB_BTREE or DB_RECNO) is to be used.
Depending on which of these is actually chosen, the final parameter,
openinfo points to a data structure which allows tailoring of the
specific interface method.
This interface is handled slightly differently in DB_File. Here is an equivalent call using DB_File.
The filename
, flags
and mode
parameters are the direct
equivalent of their dbopen() counterparts. The final parameter $DB_HASH
performs the function of both the type
and openinfo
parameters in
dbopen().
In the example above $DB_HASH is actually a reference to a hash object. DB_File has three of these pre-defined references. Apart from $DB_HASH, there is also $DB_BTREE and $DB_RECNO.
The keys allowed in each of these pre-defined references is limited to
the names used in the equivalent C structure. So, for example, the
$DB_HASH reference will only allow keys called bsize
, cachesize
,
ffactor
, hash
, lorder
and nelem
.
To change one of these elements, just assign to it like this
In order to make RECNO more compatible with Perl the array offset for all RECNO arrays begins at 0 rather than 1 as in Berkeley DB.
(char *)0 in C) in place of the filename. B
uses undef
instead of NULL to provide this functionality.
Using the Berkeley DB Interface Directly
As well as accessing Berkeley DB using a tied hash or array, it is also
possible to make direct use of most of the functions defined in the
Berkeley DB documentation.
To do this you need to remember the return value from the tie.
$db = tie %hash, DB_File, "filename"
Once you have done that, you can access the Berkeley DB API functions
directly.
$db->put($key, $value, R_NOOVERWRITE) ;
All the functions defined in dbx(3X) are available except for
close() and dbopen() itself. The DB_File interface to these
functions have been implemented to mirror the the way Berkeley DB
works. In particular note that all the functions return only a status
value. Whenever a Berkeley DB function returns data via one of its
parameters, the DB_File equivalent does exactly the same.
All the constants defined in the dbopen manpage
are also available.
Below is a list of the functions available.
-
get
-
Same as in
recno
except that the flags parameter is optional.
Remember the value associated with the key you request is returned in
the $value parameter.
-
put
-
As usual the flags parameter is optional.
If you use either the R_IAFTER or R_IBEFORE flags, the key parameter
will have the record number of the inserted key/value pair set.
-
del
-
The flags parameter is optional.
-
fd
-
As in recno.
-
seq
-
The flags parameter is optional.
Both the key and value parameters will be set.
-
sync
-
The flags parameter is optional.
EXAMPLES
It is always a lot easier to understand something when you see a real
example. So here are a few.
Using HASH
use DB_File ;
use Fcntl ;
tie %h, "DB_File", "hashed", O_RDWR|O_CREAT, 0640, $DB_HASH ;
# Add a key/value pair to the file
$h{"apple"} = "orange" ;
# Check for existence of a key
print "Exists\n" if $h{"banana"} ;
# Delete
delete $h{"apple"} ;
untie %h ;
Using BTREE
Here is sample of code which used BTREE. Just to make life more
interesting the default comparision function will not be used. Instead
a Perl sub, Compare()
, will be used to do a case insensitive
comparison.
use DB_File ;
use Fcntl ;
sub Compare
{
my ($key1, $key2) = @_ ;
"\L$key1" cmp "\L$key2" ;
}
$DB_BTREE->{compare} = 'Compare' ;
tie %h, 'DB_File', "tree", O_RDWR|O_CREAT, 0640, $DB_BTREE ;
# Add a key/value pair to the file
$h{'Wall'} = 'Larry' ;
$h{'Smith'} = 'John' ;
$h{'mouse'} = 'mickey' ;
$h{'duck'} = 'donald' ;
# Delete
delete $h{"duck"} ;
# Cycle through the keys printing them in order.
# Note it is not necessary to sort the keys as
# the btree will have kept them in order automatically.
foreach (keys %h)
{ print "$_\n" }
untie %h ;
Here is the output from the code above.
mouse
Smith
Wall
Using RECNO
use DB_File ;
use Fcntl ;
$DB_RECNO->{psize} = 3000 ;
tie @h, DB_File, "text", O_RDWR|O_CREAT, 0640, $DB_RECNO ;
# Add a key/value pair to the file
$h[0] = "orange" ;
# Check for existence of a key
print "Exists\n" if $h[1] ;
untie @h ;
Locking Databases
Concurrent access of a read-write database by several parties requires
them all to use some kind of locking. Here's an example of Tom's that
uses the
fd
method to get the file descriptor, and then a careful
open() to give something Perl will flock() for you. Run this repeatedly
in the background to watch the locks granted in proper order.
use Fcntl;
use DB_File;
use strict;
sub LOCK_SH { 1 }
sub LOCK_EX { 2 }
sub LOCK_NB { 4 }
sub LOCK_UN { 8 }
my($oldval, $fd, $db, %db, $value, $key);
$key = shift || 'default';
$value = shift || 'magic';
$value .= " $$";
$db = tie(%db, 'DB_File', '/tmp/foo.db', O_CREAT|O_RDWR, 0644)
|| die "dbcreat /tmp/foo.db $!";
$fd = $db->fd;
print "$$: db fd is $fd\n";
open(DB_FH, "+<&=$fd") || die "dup $!";
unless (flock (DB_FH, LOCK_SH | LOCK_NB)) {
print "$$: CONTENTION; can't read during write update!
Waiting for read lock ($!) ....";
unless (flock (DB_FH, LOCK_SH)) { die "flock: $!" }
}
print "$$: Read lock granted\n";
$oldval = $db{$key};
print "$$: Old value was $oldval\n";
flock(DB_FH, LOCK_UN);
unless (flock (DB_FH, LOCK_EX | LOCK_NB)) {
print "$$: CONTENTION; must have exclusive lock!
Waiting for write lock ($!) ....";
unless (flock (DB_FH, LOCK_EX)) { die "flock: $!" }
}
print "$$: Write lock granted\n";
$db{$key} = $value;
sleep 10;
flock(DB_FH, LOCK_UN);
untie %db;
close(DB_FH);
print "$$: Updated db to $key=$value\n";
HISTORY
-
0.1
-
First Release.
-
0.2
-
When DB_File is opening a database file it no longer terminates the
process if dbopen returned an error. This allows file protection
errors to be caught at run time. Thanks to Judith Grass
<grass@cybercash.com> for spotting the bug.
-
0.3
-
Added prototype support for multiple btree compare callbacks.
-
1.0
-
DB_File has been in use for over a year. To reflect that, the
version number has been incremented to 1.0.
Added complete support for multiple concurrent callbacks.
Using the push method on an empty list didn't work properly. This
has been fixed.
-
1.01
-
Fixed a core dump problem with SunOS.
The return value from TIEHASH wasn't set to NULL when dbopen returned
an error.
WARNINGS
If you happen find any other functions defined in the source for this
module that have not been mentioned in this document -- beware. I may
drop them at a moments notice.
If you cannot find any, then either you didn't look very hard or the
moment has passed and I have dropped them.
BUGS
Some older versions of Berkeley DB had problems with fixed length
records using the RECNO file format. The newest version at the time of
writing was 1.85 - this seems to have fixed the problems with RECNO.
I am sure there are bugs in the code. If you do find any, or can
suggest any enhancements, I would welcome your comments.
AVAILABILITY
Berkeley DB is available at your nearest CPAN archive (see
``CPAN'' for a list) in src/misc/db.1.85.tar.gz, or via the
host ftp.cs.berkeley.edu in /ucb/4bsd/db.tar.gz. It is not under
the GPL.
SEE ALSO
perl(1), dbopen(3), hash(3), recno(3), btree(3)
Berkeley DB is available from ftp.cs.berkeley.edu in the directory
/ucb/4bsd.
AUTHOR
The DB_File interface was written by Paul Marquess
<pmarquess@bfsec.bt.co.uk>.
Questions about the DB system itself may be addressed to Keith Bostic
<bostic@cs.berkeley.edu>.