Xenstore protocol specification ------------------------------- Xenstore implements a database which maps filename-like pathnames (also known as `keys') to values. Clients may read and write values, watch for changes, and set permissions to allow or deny access. There is a rudimentary transaction system. While xenstore and most tools and APIs are capable of dealing with arbitrary binary data as values, this should generally be avoided. Data should generally be human-readable for ease of management and debugging; xenstore is not a high-performance facility and should be used only for small amounts of control plane data. Therefore xenstore values should normally be 7-bit ASCII text strings containing bytes 0x20..0x7f only, and should not contain a trailing nul byte. (The APIs used for accessing xenstore generally add a nul when reading, for the caller's convenience.) A separate specification will detail the keys and values which are used in the Xen system and what their meanings are. (Sadly that specification currently exists only in multiple out-of-date versions.) Paths are /-separated and start with a /, just as Unix filenames. We can speak of two paths being and , which is the case if they're identical, or if is /, or if / is an initial substring of . (This includes being a child of itself.) If a particular path exists, all of its parents do too. Every existing path maps to a possibly empty value, and may also have zero or more immediate children. There is thus no particular distinction between directories and leaf nodes. However, it is conventional not to store nonempty values at nodes which also have children. The permitted character for paths set is ASCII alphanumerics and plus the four punctuation characters -/_@ (hyphen slash underscore atsign). @ should be avoided except to specify special watches (see below). Doubled slashes and trailing slashes (except to specify the root) are forbidden. The empty path is also forbidden. Paths longer than 3072 bytes are forbidden; clients specifying relative paths should keep them to within 2048 bytes. (See XENSTORE_*_PATH_MAX in xs_wire.h.) Each node has one or multiple permission entries. Permissions are granted by domain-id, the first permission entry of each node specifies the owner of the node, who always has full access to the node (read and write permission). The access rights of the first entry specify the allowed access for all domains not having a dedicated permission entry (the default is "n", removing access for all domains not explicitly added via additional permission entries). Permissions of a node can be changed by the owner of the node, the owner can only be modified by the control domain (usually domain id 0). Other permissions can be setup to allow read and/or write access for other domains. When a domain is being removed from Xenstore nodes owned by that domain will be removed together with all of those nodes' children. Communication with xenstore is via either sockets, or event channel and shared memory, as specified in io/xs_wire.h: each message in either direction is a header formatted as a struct xsd_sockmsg followed by xsd_sockmsg.len bytes of payload. The payload syntax varies according to the type field. Generally requests each generate a reply with an identical type, req_id and tx_id. However, if an error occurs, a reply will be returned with type ERROR, and only req_id and tx_id copied from the request. A caller who sends several requests may receive the replies in any order and must use req_id (and tx_id, if applicable) to match up replies to requests. (The current implementation always replies to requests in the order received but this should not be relied on.) The payload length (len field of the header) is limited to 4096 (XENSTORE_PAYLOAD_MAX) in both directions. If a client exceeds the limit, its xenstored connection will be immediately killed by xenstored, which is usually catastrophic from the client's point of view. Clients (particularly domains, which cannot just reconnect) should avoid this. Existing clients do not always contain defences against overly long payloads. Increasing xenstored's limit is therefore difficult; it would require negotiation with the client, and obviously would make parts of xenstore inaccessible to some clients. In any case passing bulk data through xenstore is not recommended as the performance properties are poor. ---------- Xenstore protocol details - introduction ---------- The payload syntax and semantics of the requests and replies are described below. In the payload syntax specifications we use the following notations: | A nul (zero) byte. A string guaranteed not to contain any nul bytes. Binary data (which may contain zero or more nul bytes) |* Zero or more strings each followed by a trailing nul |+ One or more strings each followed by a trailing nul ? Reserved value (may not contain nuls) ?? Reserved value (may contain nuls) Except as otherwise noted, reserved values are believed to be sent as empty strings by all current clients. Clients should not send nonempty strings for reserved values; those parts of the protocol may be used for extension in the future. Error replies are as follows: ERROR E| Where E is the name of an errno value listed in io/xs_wire.h. Note that the string name is transmitted, not a numeric value. Where no reply payload format is specified below, success responses have the following payload: OK| Values commonly included in payloads include: Specifies a path in the hierarchical key structure. If starts with a / it simply represents that path. is allowed not to start with /, in which case the caller must be a domain (rather than connected via a socket) and the path is taken to be relative to /local/domain/ (eg, `x/y' sent by domain 3 would mean `/local/domain/3/x/y'). Integer domid, represented as decimal number 0..65535. Parsing errors and values out of range generally go undetected. The special DOMID_... values (see xen.h) are represented as integers; unless otherwise specified it is an error not to specify a real domain id. The following are the actual type values, including the request and reply payloads as applicable: ---------- Database read, write and permissions operations ---------- READ | WRITE | Store and read the octet string at . WRITE creates any missing parent paths, with empty values. MKDIR | Ensures that the exists, by necessary by creating it and any missing parents with empty values. If or any parent already exists, its value is left unchanged. RM | Ensures that the does not exist, by deleting it and all of its children. It is not an error if does not exist, but it _is_ an error if 's immediate parent does not exist either. DIRECTORY | |* Gives a list of the immediate children of , as only the leafnames. The resulting children are each named /. DIRECTORY_PART | ||* Same as DIRECTORY, but to be used for children lists longer than XENSTORE_PAYLOAD_MAX. Input are and the byte offset into the list of children to return. Return values are the generation count of the node (to be used to ensure the node hasn't changed between two reads: being the same for multiple reads guarantees the node hasn't changed) and the list of children starting at the specified of the complete list. GET_PERMS | |+ SET_PERMS ||+? is one of the following w write only r read only b both read and write n no access See https://wiki.xen.org/wiki/XenBus section `Permissions' for details of the permissions system. It is possible to set permissions for the special watch paths "@introduceDomain" and "@releaseDomain" to enable receiving those watches in unprivileged domains. ---------- Watches ---------- WATCH ||[|]? Adds a watch. When a is modified (including path creation, removal, contents change or permissions change) this generates an event on the changed . Changes made in transactions cause an event only if and when committed. Each occurring event is matched against all the watches currently set up, and each matching watch results in a WATCH_EVENT message (see below). The event's path matches the watch's if it is an child of . This match can be limited by specifying (a decimal value of 0 or larger): it denotes the directory levels below to consider for a match ("0" would not match for a child of , "1" would match only for a direct child, etc.). can be a to watch or @. In the latter case may have any syntax but it matches (according to the rules above) only the following special events which are invented by xenstored: @introduceDomain occurs on INTRODUCE @releaseDomain occurs on any domain crash or shutdown, and also on RELEASE and domain destruction events are sent to privileged callers or explicitly via SET_PERMS enabled domains only. The semantics for a specification of differ for generating events: specifying "1" will report the related domid by using @/ for the reported path. Other values are not supported. For @releaseDomain it is possible to watch only for a specific domain by specifying @releaseDomain/ for the path. When a watch is first set up it is triggered once straight away, with equal to . Watches may be triggered spuriously. The tx_id in a WATCH request is ignored. Watches are supposed to be restricted by the permissions system but in practice the implementation is imperfect. Applications should not rely on being sent a notification for paths that they cannot read; however, an application may rely on being sent a watch when a path which it _is_ able to read is deleted even if that leaves only a nonexistent unreadable parent. A notification may omitted if a node's permissions are changed so as to make it unreadable, in which case future notifications may be suppressed (and if the node is later made readable, some notifications may have been lost). WATCH_EVENT || Unsolicited `reply' generated for matching modification events as described above. req_id and tx_id are both 0. is the event's path, ie the actual path that was modified; however if the event was the recursive removal of an parent of , is just (rather than the actual path which was removed). So is a child of , regardless. Iff for the watch was specified as a relative pathname, the path will also be relative (with the same base, obviously). UNWATCH ||? RESET_WATCHES | Reset all watches and transactions of the caller. ---------- Transactions ---------- TRANSACTION_START | | is an opaque uint32_t allocated by xenstored represented as unsigned decimal. After this, transaction may be referenced by using (as 32-bit binary) in the tx_id request header field. When transaction is started whole db is copied; reads and writes happen on the copy. It is not legal to send non-0 tx_id in TRANSACTION_START. TRANSACTION_END T| TRANSACTION_END F| tx_id must refer to existing transaction. After this request the tx_id is no longer valid and may be reused by xenstore. If F, the transaction is discarded. If T, it is committed: if there were any other intervening writes then our END gets get EAGAIN. The plan is that in the future only intervening `conflicting' writes cause EAGAIN, meaning only writes or other commits which changed paths which were read or written in the transaction at hand. ---------- Domain management and xenstored communications ---------- INTRODUCE |||? Notifies xenstored to communicate with this domain. INTRODUCE is currently only used by xend (during domain startup and various forms of restore and resume), and xenstored prevents its use other than by dom0. must be a real domain id (not 0 and not a special DOMID_... value). must be a page in that domain represented in signed decimal (!). must be event channel is an unbound event channel in (likewise in decimal), on which xenstored will call bind_interdomain. Violations of these rules may result in undefined behaviour; for example passing a high-bit-set 32-bit gfn as an unsigned decimal will attempt to use 0x7fffffff instead (!). RELEASE | Manually requests that xenstored disconnect from the domain. The event channel is unbound at the xenstored end and the page unmapped. If the domain is still running it won't be able to communicate with xenstored. NB that xenstored will in any case detect domain destruction and disconnect by itself. xenstored prevents the use of RELEASE other than by dom0. GET_DOMAIN_PATH | | Returns the domain's base path, as is used for relative transactions: ie, /local/domain/ (with normalised). The answer will be useless unless is a real domain id. IS_DOMAIN_INTRODUCED | T| or F| Returns T if xenstored is in communication with the domain: ie, if INTRODUCE for the domain has not yet been followed by domain destruction or explicit RELEASE. RESUME | Arranges that @releaseDomain events will once more be generated when the domain becomes shut down. This might have to be used if a domain were to be shut down (generating one @releaseDomain) and then subsequently restarted, since the state-sensitive algorithm in xenstored will not otherwise send further watch event notifications if the domain were to be shut down again. This command will be issued in place such as resume because Xen will "shutdown" the domain on suspend. xenstored prevents the use of RESUME other than by dom0. SET_TARGET || Notifies xenstored that domain is targeting domain . This grants domain full access to paths owned by . Domain also inherits all permissions granted to on all other paths. This allows to behave as if it were dom0 when modifying paths related to . xenstored prevents the use of SET_TARGET other than by dom0. GET_FEATURE [|] | SET_FEATURE || Returns or sets the contents of the "feature" field copied to offset 2064 of the Xenstore ring page of the domain specified by . is a decimal number being a logical or of the feature bits as defined in docs/misc/xenstore-ring.txt. Trying to set a bit for a feature not being supported by the running Xenstore will be denied. Providing no with the GET_FEATURE command will return the features which are supported by Xenstore. SET_FEATURE for a domain will be rejected after the INTRODUCE command for this domain has been sent to xenstored. xenstored prevents the use of GET_FEATURE and SET_FEATURE other than by dom0. GET_QUOTA [[|]|] | SET_QUOTA [|]|| Returns or sets a quota value for the domain being specified by . Omitting will return or set the global quota values, which are the default values for new domains. is one of "nodes", "watches", "transactions", "node-size", "permissions", or any other implementation defined value. For GET_QUOTA it is possible to omit the parameter together with the parameter, which will return a single string of all supported values separated by blanks. is a decimal number specifying the quota value, with "0" having the special meaning of quota checks being disabled. The initial quota settings for a domain are the global ones of Xenstore. xenstored prevents the use of GET_QUOTA and SET_QUOTA other than by dom0. ---------- Miscellaneous ---------- CONTROL |[|] Send a control command with optional parameters () to Xenstore daemon. The set of commands and their semantics is implementation specific and is likely to change from one Xen version to the next. Out-of-tree users will encounter compatibility issues. Current commands are: check checks xenstored innards live-update||+ perform a live-update of the Xenstore daemon, only to be used via xenstore-control command. are implementation specific and are used for different steps of the live-update processing. Currently supported are: -f specify new daemon binary -b specify size of new stubdom binary -d transfer chunk of new stubdom binary -c specify new command line to use -s [-t ] [-F] start live update process (-t specifies timeout in seconds to wait for active transactions to finish, default is 60 seconds; -F will force live update to happen even with running transactions after timeout elapsed) -a abort live update handling All sub-options will return "OK" in case of success or an error string in case of failure. -s can return "BUSY" in case of an active transaction, a retry of -s can be done in that case. log|[on|off|+|-] without parameters: show possible log switches on: turn xenstore logging on off: turn xenstore logging off +: activates log entries for , -: deactivates log entries for logfile| log to specified file memreport|[] print memory statistics to logfile (no specified) or to specific file print| print to syslog (xenstore runs as daemon) or to console (xenstore runs as stubdom) quota|[set ||max [-r]] without parameters: print the current quota settings with "set ": set the quota to new value (The admin should make sure all the domain usage is below the quota. If it is not, then Xenstored may continue to handle requests from the domain as long as the resource violating the new quota setting isn't increased further) with "": print quota related accounting data for the domain with "max [-r]": show global per-domain maximum values of all unprivileged domains, optionally reset the values by adding "-r" quota-soft|[set ] like the "quota" command, but for soft-quota. help return list of supported commands for CONTROL DEBUG Deprecated, now named CONTROL