jpt - Junoplay Table

Introduction

jpt is a database designed for use from applications. Its primary component is libjpt, which can be easily linked into C++ and C programs.

Each database is represented by a single sparse table, which can have up to 2³² columns and an "unlimited" number of rows. Each cell can hold up to 4 GB of data, although this number is limited by your computer's RAM.

The database employs PATRICIA tries and bloom filters for fast data retrieval. Newly inserted data is buffered in a splay tree (as well as a linear transaction log file) for fast data storage.

Unlike Berkeley DB and SQL databases, jpt supports the "append" operation (along with "insert", "replace" and "get"), making it suitable for building indexes fast.

Every cell has a 64 bit timestamp denoting the number of microseconds since 1970-01-01 00:00 UTC.

Cells are clustered by column rather than my row, so that scanning a single column is fast.

The library is always thread-safe, allowing multiple simultaneous readers or one writer.

Download

Latest development version: git://junoplay.com/jpt/

Date	Version	Download link	Changes
2008-11-13	1.0.0	jpt-1.0.0.tar.gz	First release

Known bugs

Database files are not portable between big-endian and little-endian machines. To move between big-endian and little-endian, you need to save a backup on one architecture and restore it on the other.
If you decrease your application's memtable size, you risk losing data if you don't compact the database first (truncating the log).

Example usage

For clarity, here's the example program without #includes and without error reporting:

int main(int argc, char** argv)
{
  struct JPT_info* db;
  const char* wr_value = "horses are awesome";
  char* rd_value;
  size_t rd_value_size;

  db = jpt_init("testdata", 8 * 1024 * 1024, 0);

  jpt_insert(db, "my-row", "my-column", wr_value, strlen(wr_value), JPT_REPLACE);

  jpt_get(db, "my-row", "my-column", (void*) &rd_value, &rd_value_size);

  fprintf(stderr, "The value at my-row/my-column is: '%s'\n", rd_value);
  free(rd_value);

  jpt_close(db);

  return EXIT_SUCCESS;
}

And now, the version with #includes and error reporting:

#include <errno.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>

#include <jpt.h>

int main(int argc, char** argv)
{
  struct JPT_info* db;
  const char* wr_value = "horses are awesome";
  char* rd_value;
  size_t rd_value_size;

  /* Open the database, create it if it does not exist.
   *
   * Allocate 8 MB for the memtable, meaning a new disktable will be created
   * for every 8 MB of inserted data (including overhead).
   */
  db = jpt_init("testdata", 8 * 1024 * 1024, 0);

  if(!db)
  {
    fprintf(stderr, "Failed to open database 'testdata': %s\n", jpt_last_error());

    return EXIT_FAILURE;
  }

  /* Insert a value into the database.
   *
   * JPT_REPLACE means the cell's value will be replaced if it exists.
   */
  if(-1 == jpt_insert(db, "my-row", "my-column", wr_value, strlen(wr_value), JPT_REPLACE))
  {
    fprintf(stderr, "Insert failed: %s\n", jpt_last_error());

    return EXIT_FAILURE;
  }

  /* Fetch a value from the database */
  if(-1 == jpt_get(db, "my-row", "my-column", (void*) &rd_value, &rd_value_size))
  {
    fprintf(stderr, "Get failed: %s\n", jpt_last_error());

    return EXIT_FAILURE;
  }

  /* Note: jpt_get() always adds a NUL byte past the end of the value buffer
   * so that, if the value is a text string, it can easily be passed to printf.
   */

  fprintf(stderr, "The value at my-row/my-column is: '%s'\n", rd_value);
  free(rd_value);

  jpt_close(db);

  return EXIT_SUCCESS;
}

Report bugs to morten@rashbox.org, and please start your subject line with [jpt].