Table of Contents

DataStore

Stale page

The contents of this page must be revised to reflect the current state of MINIX3.

The MINIX 3 DataStore (DS) server allows components to back up data and retrieve it after a restart and allows them to exchange data indirectly. For each publish request, an entry is stored in DS.

Currently DS supports 5 types of data: U32 (unsigned int), STRING (null-terminated string), MEM (memory range), MAP (mapped memory range), and LABEL (as name server).

For U32, STRING and MEM, when published, DS stores a copy of the data, and programs can retrieve this copy later. U32 and STRING are short enough that the data can be passed directly in a message, so they are more efficient. A MEM value has to be granted access first.

For MAP, DS maps the memory range to DS's address space, which gives a real-time view of that range in a given process. What's more, snapshots can be made any time.

Entries in the 4 types above are stored under a component-specified identifier, which is a string, called key-name. However, the key-name is not unique because of different types. This means that there may be 2 entries will same key-name, but different types. Therefore components have to use both the key-name and type to retrieve/delete the stored data.

For LABEL, it builds an one-to-one (bijection) mapping between a name and a number to allow it to be used as a name server. Because of the bijection, there's a difference with other types; the LABEL entries are not identified by key-name/type. There won't be 2 entries with same name, or with same number. This assures that you can retrieve the name by its number, and retrieve the number by its name.

U32, STRING and MEM entries can be overwritten by key-name; LABEL can be overwritten by label-name or label-num; while MAP can't be overwritten.

All types of entries can be subscribed, through the key-name (or the string if LABEL type). If the subscribed entry is created, changed or deleted, DS will notify the subscriber. An exception is the MAP type. Since its data changes invisibly to DS, the changes will not cause any notifications.

After getting a notification, the subscriber can check which entry was changed, which will give the key-name and type. Then you can use them to retrieve the changed data. Note that one notification may end up being delivered when multiple entries have changed, so the subscriber must call ds_check in a loop until ENOENT is returned.

API

All function prototype and flags (described in the following section) are defined in include/minix/ds.h.

Brief view

U32 functions

ds_publish_u32

int ds_publish_u32(const char *ds_name, u32_t value, int flags);

Publish an unsigned int. If exists, overwrite it.

PARAMETERS:

RETURN VALUE:

ds_retrieve_u32

int ds_retrieve_u32(const char *ds_name, u32_t *value);

Retrieve an unsigned int.

PARAMETERS:

RETURN VALUE:

ds_delete_u32

int ds_delete_u32(const char *ds_name);

Delete an unsigned int.

PARAMETERS:

RETURN VALUE:

STRING functions

ds_publish_str

int ds_publish_str(const char *ds_name, const char *string, int flags);

Publish a string.

PARAMETERS:

RETURN VALUE:

ds_retrieve_str

int ds_retrieve_str(const char *ds_name, char *value, size_t length);

Retrieve a string.

PARAMETERS:

RETURN VALUE:

/!\ WARNING: the supplied buffer must be at least one byte larger than the given length, for the final null byte

ds_delete_str

int ds_delete_str(const char *ds_name);

Delete a string.

PARAMETERS:

RETURN VALUE:

MEM functions

ds_publish_mem

int ds_publish_mem(const char *ds_name, void *vaddr, size_t length, int flags);

Publish a memory range. If specified, it can overwrite an older entry with the new memory range, regardless of its length.

PARAMETERS:

RETURN VALUE:

ds_retrieve_mem

int ds_retrieve_mem(const char *ds_name, char *vaddr, size_t *length);

Retrieve (part of) a memory range.

PARAMETERS:

RETURN VALUE:

ds_delete_mem

int ds_delete_mem(const char *ds_name);

Delete a memory range.

PARAMETERS:

RETURN VALUE:

MAP functions

ds_publish_map

int ds_publish_map(const char *ds_name, void *vaddr, size_t length, int flags);

Publish a part of a memory range, using a mapping.

PARAMETERS:

RETURN VALUE:

ds_snapshot_map

int ds_snapshot_map(const char *ds_name, int *nr_snapshot);

Make a snapshot of a mapped memory range.

PARAMETERS:

RETURN VALUE:

ds_retrieve_map

int ds_retrieve_map(const char *ds_name, char *vaddr, size_t *length, int nr_snapshot);

Retrieve (part of) a mapped memory range or snapshot.

PARAMETERS:

RETURN VALUE:

ds_delete_map

int ds_delete_map(const char *ds_name);

Delete a mapped memory range, and its snapshots, if any.

PARAMETERS:

RETURN VALUE:

LABEL functions

ds_publish_label

int ds_publish_label(const char *ds_name, endpoint_t value, int flags);

Publish a label, (name, number);

PARAMETERS:

RETURN VALUE:

ds_retrieve_label_endpt

int ds_retrieve_label_num(const char *ds_name, endpoint_t *num);

Retrieve a label's endpoint.

PARAMETERS:

RETURN VALUE:

ds_retrieve_label_name

int ds_retrieve_label_name(char *ds_name, endpoint_t num);

Retrieve a label's name.

PARAMETERS:

RETURN VALUE:

ds_delete_label

int ds_delete_label(const char *ds_name);

Delete a label.

PARAMETERS:

RETURN VALUE:

SUBSCRIBE functions

ds_subscribe

int ds_subscribe(const char *regex, int flags);

Subscribe to an entry's changes.

PARAMETERS:

RETURN VALUE:

ds_check

int ds_check(char *ds_name, int *type);

Check which subscribed memory range has changed. This function will give you which entry has changed, then you can use the returned key and type to call ds_retrieve_xxx to get the changed data.

PARAMETERS:

RETURN VALUE:

Flags

DSF_PRIV_RETRIEVE, DSF_PRIV_OVERWRITE, DSF_PRIV_SUBSCRIB, DSF_PRIV_SNAPSHOT

Set private. If one of these flags is set, then this entry will not be retrieved, overwritten, subscribed, or snapshot by other processes.  Used when publishing data.

DSF_TYPE_U32, DSF_TYPE_STR, DSF_TYPE_MEM, DSF_TYPE_MAP, DSF_TYPE_LABEL

  1. Used when subscribing to data.

DSF_OVERWRITE

If there is an entry identified by the same key-name, if set, overwrite it.  Used when publishing data.

DSF_INITIAL

If set, check whether there is any entry matching this subscription instantly.  Used when subscribing to data.

Test

There are 2 system processes to test DataStore, dstest and subs. They are located in /usr/src/test/ds.

dstest tests all new APIs of DS, except ds_subscribe() and ds_check(). test_u32, test_str, test_mem, test_label and test_map test U32, STR, MEM, LABEL and MAP type respectively.

Invalid invocations are tested too. If the same erroneous parameters are tested in former type, it will not be tested in a new type. For example, publishing an entry with same label-name, but without DSF_OVERWRITE set is tested in test_u32, so it's not tested in other types again. New type tests only test new features, which belong to the new type only.

subs tests ds_subscribe() and ds_check(). This server subscribes the U32 (identified by 'test_u32') in dstest. When dstest runs, subs catches it.

See the README with test file for more details.

Others

DS dump

Press Shift+F8 to show the data store dump.

Message

The DS request message is mess_2 ::

typedef struct {
	int m2i1, m2i2, m2i3;
	long m2l1, m2l2;
	char *m2p1; short m2s1;
} mess_2;

Page Table

For storing memory, we don't know how much memory DS needs. So DS should have its own page table, to allocate memory dynamically.