view tools/xenstore/talloc_guide.txt @ 21067:b4a1832a916f

Update Xen version to 4.0.0-rc6
author Keir Fraser <>
date Tue Mar 09 18:18:05 2010 +0000 (2010-03-09)
parents 6d3e8f90c2df
line source
1 Using talloc in Samba4
2 ----------------------
4 Andrew Tridgell
5 September 2004
7 The most current version of this document is available at
10 If you are used to talloc from Samba3 then please read this carefully,
11 as talloc has changed a lot.
13 The new talloc is a hierarchical, reference counted memory pool system
14 with destructors. Quite a mounthful really, but not too bad once you
15 get used to it.
17 Perhaps the biggest change from Samba3 is that there is no distinction
18 between a "talloc context" and a "talloc pointer". Any pointer
19 returned from talloc() is itself a valid talloc context. This means
20 you can do this:
22 struct foo *X = talloc(mem_ctx, struct foo);
23 X->name = talloc_strdup(X, "foo");
25 and the pointer X->name would be a "child" of the talloc context "X"
26 which is itself a child of mem_ctx. So if you do talloc_free(mem_ctx)
27 then it is all destroyed, whereas if you do talloc_free(X) then just X
28 and X->name are destroyed, and if you do talloc_free(X->name) then
29 just the name element of X is destroyed.
31 If you think about this, then what this effectively gives you is an
32 n-ary tree, where you can free any part of the tree with
33 talloc_free().
35 If you find this confusing, then I suggest you run the testsuite to
36 watch talloc in action. You may also like to add your own tests to
37 testsuite.c to clarify how some particular situation is handled.
40 Performance
41 -----------
43 All the additional features of talloc() over malloc() do come at a
44 price. We have a simple performance test in Samba4 that measures
45 talloc() versus malloc() performance, and it seems that talloc() is
46 about 10% slower than malloc() on my x86 Debian Linux box. For Samba,
47 the great reduction in code complexity that we get by using talloc
48 makes this worthwhile, especially as the total overhead of
49 talloc/malloc in Samba is already quite small.
52 talloc API
53 ----------
55 The following is a complete guide to the talloc API. Read it all at
56 least twice.
59 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
60 (type *)talloc(const void *context, type);
62 The talloc() macro is the core of the talloc library. It takes a
63 memory context and a type, and returns a pointer to a new area of
64 memory of the given type.
66 The returned pointer is itself a talloc context, so you can use it as
67 the context argument to more calls to talloc if you wish.
69 The returned pointer is a "child" of the supplied context. This means
70 that if you talloc_free() the context then the new child disappears as
71 well. Alternatively you can free just the child.
73 The context argument to talloc() can be NULL, in which case a new top
74 level context is created.
77 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
78 void *talloc_size(const void *context, size_t size);
80 The function talloc_size() should be used when you don't have a
81 convenient type to pass to talloc(). Unlike talloc(), it is not type
82 safe (as it returns a void *), so you are on your own for type checking.
85 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
86 int talloc_free(void *ptr);
88 The talloc_free() function frees a piece of talloc memory, and all its
89 children. You can call talloc_free() on any pointer returned by
90 talloc().
92 The return value of talloc_free() indicates success or failure, with 0
93 returned for success and -1 for failure. The only possible failure
94 condition is if the pointer had a destructor attached to it and the
95 destructor returned -1. See talloc_set_destructor() for details on
96 destructors.
98 If this pointer has an additional parent when talloc_free() is called
99 then the memory is not actually released, but instead the most
100 recently established parent is destroyed. See talloc_reference() for
101 details on establishing additional parents.
103 For more control on which parent is removed, see talloc_unlink()
105 talloc_free() operates recursively on its children.
108 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
109 int talloc_free_children(void *ptr);
111 The talloc_free_children() walks along the list of all children of a
112 talloc context and talloc_free()s only the children, not the context
113 itself.
116 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
117 void *talloc_reference(const void *context, const void *ptr);
119 The talloc_reference() function makes "context" an additional parent
120 of "ptr".
122 The return value of talloc_reference() is always the original pointer
123 "ptr", unless talloc ran out of memory in creating the reference in
124 which case it will return NULL (each additional reference consumes
125 around 48 bytes of memory on intel x86 platforms).
127 If "ptr" is NULL, then the function is a no-op, and simply returns NULL.
129 After creating a reference you can free it in one of the following
130 ways:
132 - you can talloc_free() any parent of the original pointer. That
133 will reduce the number of parents of this pointer by 1, and will
134 cause this pointer to be freed if it runs out of parents.
136 - you can talloc_free() the pointer itself. That will destroy the
137 most recently established parent to the pointer and leave the
138 pointer as a child of its current parent.
140 For more control on which parent to remove, see talloc_unlink()
143 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
144 int talloc_unlink(const void *context, const void *ptr);
146 The talloc_unlink() function removes a specific parent from ptr. The
147 context passed must either be a context used in talloc_reference()
148 with this pointer, or must be a direct parent of ptr.
150 Note that if the parent has already been removed using talloc_free()
151 then this function will fail and will return -1. Likewise, if "ptr"
152 is NULL, then the function will make no modifications and return -1.
154 Usually you can just use talloc_free() instead of talloc_unlink(), but
155 sometimes it is useful to have the additional control on which parent
156 is removed.
159 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
160 void talloc_set_destructor(const void *ptr, int (*destructor)(void *));
162 The function talloc_set_destructor() sets the "destructor" for the
163 pointer "ptr". A destructor is a function that is called when the
164 memory used by a pointer is about to be released. The destructor
165 receives the pointer as an argument, and should return 0 for success
166 and -1 for failure.
168 The destructor can do anything it wants to, including freeing other
169 pieces of memory. A common use for destructors is to clean up
170 operating system resources (such as open file descriptors) contained
171 in the structure the destructor is placed on.
173 You can only place one destructor on a pointer. If you need more than
174 one destructor then you can create a zero-length child of the pointer
175 and place an additional destructor on that.
177 To remove a destructor call talloc_set_destructor() with NULL for the
178 destructor.
180 If your destructor attempts to talloc_free() the pointer that it is
181 the destructor for then talloc_free() will return -1 and the free will
182 be ignored. This would be a pointless operation anyway, as the
183 destructor is only called when the memory is just about to go away.
186 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
187 void talloc_increase_ref_count(const void *ptr);
189 The talloc_increase_ref_count(ptr) function is exactly equivalent to:
191 talloc_reference(NULL, ptr);
193 You can use either syntax, depending on which you think is clearer in
194 your code.
197 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
198 void talloc_set_name(const void *ptr, const char *fmt, ...);
200 Each talloc pointer has a "name". The name is used principally for
201 debugging purposes, although it is also possible to set and get the
202 name on a pointer in as a way of "marking" pointers in your code.
204 The main use for names on pointer is for "talloc reports". See
205 talloc_report() and talloc_report_full() for details. Also see
206 talloc_enable_leak_report() and talloc_enable_leak_report_full().
208 The talloc_set_name() function allocates memory as a child of the
209 pointer. It is logically equivalent to:
210 talloc_set_name_const(ptr, talloc_asprintf(ptr, fmt, ...));
212 Note that multiple calls to talloc_set_name() will allocate more
213 memory without releasing the name. All of the memory is released when
214 the ptr is freed using talloc_free().
217 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
218 void talloc_set_name_const(const void *ptr, const char *name);
220 The function talloc_set_name_const() is just like talloc_set_name(),
221 but it takes a string constant, and is much faster. It is extensively
222 used by the "auto naming" macros, such as talloc_p().
224 This function does not allocate any memory. It just copies the
225 supplied pointer into the internal representation of the talloc
226 ptr. This means you must not pass a name pointer to memory that will
227 disappear before the ptr is freed with talloc_free().
230 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
231 void *talloc_named(const void *context, size_t size, const char *fmt, ...);
233 The talloc_named() function creates a named talloc pointer. It is
234 equivalent to:
236 ptr = talloc_size(context, size);
237 talloc_set_name(ptr, fmt, ....);
240 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
241 void *talloc_named_const(const void *context, size_t size, const char *name);
243 This is equivalent to:
245 ptr = talloc_size(context, size);
246 talloc_set_name_const(ptr, name);
249 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
250 const char *talloc_get_name(const void *ptr);
252 This returns the current name for the given talloc pointer. See
253 talloc_set_name() for details.
256 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
257 void *talloc_init(const char *fmt, ...);
259 This function creates a zero length named talloc context as a top
260 level context. It is equivalent to:
262 talloc_named(NULL, 0, fmt, ...);
265 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
266 void *talloc_new(void *ctx);
268 This is a utility macro that creates a new memory context hanging
269 off an exiting context, automatically naming it "talloc_new: __location__"
270 where __location__ is the source line it is called from. It is
271 particularly useful for creating a new temporary working context.
274 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
275 (type *)talloc_realloc(const void *context, void *ptr, type, count);
277 The talloc_realloc() macro changes the size of a talloc
278 pointer. The "count" argument is the number of elements of type "type"
279 that you want the resulting pointer to hold.
281 talloc_realloc() has the following equivalences:
283 talloc_realloc(context, NULL, type, 1) ==> talloc(context, type);
284 talloc_realloc(context, NULL, type, N) ==> talloc_array(context, type, N);
285 talloc_realloc(context, ptr, type, 0) ==> talloc_free(ptr);
287 The "context" argument is only used if "ptr" is not NULL, otherwise it
288 is ignored.
290 talloc_realloc() returns the new pointer, or NULL on failure. The call
291 will fail either due to a lack of memory, or because the pointer has
292 more than one parent (see talloc_reference()).
295 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
296 void *talloc_realloc_size(const void *context, void *ptr, size_t size);
298 the talloc_realloc_size() function is useful when the type is not
299 known so the typesafe talloc_realloc() cannot be used.
302 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
303 void *talloc_steal(const void *new_ctx, const void *ptr);
305 The talloc_steal() function changes the parent context of a talloc
306 pointer. It is typically used when the context that the pointer is
307 currently a child of is going to be freed and you wish to keep the
308 memory for a longer time.
310 The talloc_steal() function returns the pointer that you pass it. It
311 does not have any failure modes.
313 NOTE: It is possible to produce loops in the parent/child relationship
314 if you are not careful with talloc_steal(). No guarantees are provided
315 as to your sanity or the safety of your data if you do this.
317 talloc_steal (new_ctx, NULL) will return NULL with no sideeffects.
319 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
320 off_t talloc_total_size(const void *ptr);
322 The talloc_total_size() function returns the total size in bytes used
323 by this pointer and all child pointers. Mostly useful for debugging.
325 Passing NULL is allowed, but it will only give a meaningful result if
326 talloc_enable_leak_report() or talloc_enable_leak_report_full() has
327 been called.
330 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
331 off_t talloc_total_blocks(const void *ptr);
333 The talloc_total_blocks() function returns the total memory block
334 count used by this pointer and all child pointers. Mostly useful for
335 debugging.
337 Passing NULL is allowed, but it will only give a meaningful result if
338 talloc_enable_leak_report() or talloc_enable_leak_report_full() has
339 been called.
342 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
343 void talloc_report(const void *ptr, FILE *f);
345 The talloc_report() function prints a summary report of all memory
346 used by ptr. One line of report is printed for each immediate child of
347 ptr, showing the total memory and number of blocks used by that child.
349 You can pass NULL for the pointer, in which case a report is printed
350 for the top level memory context, but only if
351 talloc_enable_leak_report() or talloc_enable_leak_report_full() has
352 been called.
355 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
356 void talloc_report_full(const void *ptr, FILE *f);
358 This provides a more detailed report than talloc_report(). It will
359 recursively print the ensire tree of memory referenced by the
360 pointer. References in the tree are shown by giving the name of the
361 pointer that is referenced.
363 You can pass NULL for the pointer, in which case a report is printed
364 for the top level memory context, but only if
365 talloc_enable_leak_report() or talloc_enable_leak_report_full() has
366 been called.
369 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
370 void talloc_enable_leak_report(void);
372 This enables calling of talloc_report(NULL, stderr) when the program
373 exits. In Samba4 this is enabled by using the --leak-report command
374 line option.
376 For it to be useful, this function must be called before any other
377 talloc function as it establishes a "null context" that acts as the
378 top of the tree. If you don't call this function first then passing
379 NULL to talloc_report() or talloc_report_full() won't give you the
380 full tree printout.
382 Here is a typical talloc report:
384 talloc report on 'null_context' (total 267 bytes in 15 blocks)
385 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks
386 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks
387 iconv(UTF8,CP850) contains 42 bytes in 2 blocks
388 libcli/auth/spnego_parse.c:55 contains 31 bytes in 2 blocks
389 iconv(CP850,UTF8) contains 42 bytes in 2 blocks
390 iconv(UTF8,UTF-16LE) contains 45 bytes in 2 blocks
391 iconv(UTF-16LE,UTF8) contains 45 bytes in 2 blocks
394 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
395 void talloc_enable_leak_report_full(void);
397 This enables calling of talloc_report_full(NULL, stderr) when the
398 program exits. In Samba4 this is enabled by using the
399 --leak-report-full command line option.
401 For it to be useful, this function must be called before any other
402 talloc function as it establishes a "null context" that acts as the
403 top of the tree. If you don't call this function first then passing
404 NULL to talloc_report() or talloc_report_full() won't give you the
405 full tree printout.
407 Here is a typical full report:
409 full talloc report on 'root' (total 18 bytes in 8 blocks)
410 p1 contains 18 bytes in 7 blocks (ref 0)
411 r1 contains 13 bytes in 2 blocks (ref 0)
412 reference to: p2
413 p2 contains 1 bytes in 1 blocks (ref 1)
414 x3 contains 1 bytes in 1 blocks (ref 0)
415 x2 contains 1 bytes in 1 blocks (ref 0)
416 x1 contains 1 bytes in 1 blocks (ref 0)
419 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
420 void talloc_enable_null_tracking(void);
422 This enables tracking of the NULL memory context without enabling leak
423 reporting on exit. Useful for when you want to do your own leak
424 reporting call via talloc_report_null_full();
427 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
428 (type *)talloc_zero(const void *ctx, type);
430 The talloc_zero() macro is equivalent to:
432 ptr = talloc(ctx, type);
433 if (ptr) memset(ptr, 0, sizeof(type));
436 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
437 void *talloc_zero_size(const void *ctx, size_t size)
439 The talloc_zero_size() function is useful when you don't have a known type
442 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
443 void *talloc_memdup(const void *ctx, const void *p, size_t size);
445 The talloc_memdup() function is equivalent to:
447 ptr = talloc_size(ctx, size);
448 if (ptr) memcpy(ptr, p, size);
451 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
452 char *talloc_strdup(const void *ctx, const char *p);
454 The talloc_strdup() function is equivalent to:
456 ptr = talloc_size(ctx, strlen(p)+1);
457 if (ptr) memcpy(ptr, p, strlen(p)+1);
459 This functions sets the name of the new pointer to the passed
460 string. This is equivalent to:
461 talloc_set_name_const(ptr, ptr)
463 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
464 char *talloc_strndup(const void *t, const char *p, size_t n);
466 The talloc_strndup() function is the talloc equivalent of the C
467 library function strndup()
469 This functions sets the name of the new pointer to the passed
470 string. This is equivalent to:
471 talloc_set_name_const(ptr, ptr)
474 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
475 char *talloc_vasprintf(const void *t, const char *fmt, va_list ap);
477 The talloc_vasprintf() function is the talloc equivalent of the C
478 library function vasprintf()
481 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
482 char *talloc_asprintf(const void *t, const char *fmt, ...);
484 The talloc_asprintf() function is the talloc equivalent of the C
485 library function asprintf()
487 This functions sets the name of the new pointer to the passed
488 string. This is equivalent to:
489 talloc_set_name_const(ptr, ptr)
492 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
493 char *talloc_asprintf_append(char *s, const char *fmt, ...);
495 The talloc_asprintf_append() function appends the given formatted
496 string to the given string.
499 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
500 (type *)talloc_array(const void *ctx, type, uint_t count);
502 The talloc_array() macro is equivalent to:
504 (type *)talloc_size(ctx, sizeof(type) * count);
506 except that it provides integer overflow protection for the multiply,
507 returning NULL if the multiply overflows.
510 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
511 void *talloc_array_size(const void *ctx, size_t size, uint_t count);
513 The talloc_array_size() function is useful when the type is not
514 known. It operates in the same way as talloc_array(), but takes a size
515 instead of a type.
518 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
519 void *talloc_realloc_fn(const void *ctx, void *ptr, size_t size);
521 This is a non-macro version of talloc_realloc(), which is useful
522 as libraries sometimes want a ralloc function pointer. A realloc()
523 implementation encapsulates the functionality of malloc(), free() and
524 realloc() in one call, which is why it is useful to be able to pass
525 around a single function pointer.
528 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
529 void *talloc_autofree_context(void);
531 This is a handy utility function that returns a talloc context
532 which will be automatically freed on program exit. This can be used
533 to reduce the noise in memory leak reports.
536 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
537 void *talloc_check_name(const void *ptr, const char *name);
539 This function checks if a pointer has the specified name. If it does
540 then the pointer is returned. It it doesn't then NULL is returned.
543 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
544 (type *)talloc_get_type(const void *ptr, type);
546 This macro allows you to do type checking on talloc pointers. It is
547 particularly useful for void* private pointers. It is equivalent to
548 this:
550 (type *)talloc_check_name(ptr, #type)
553 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
554 talloc_set_type(const void *ptr, type);
556 This macro allows you to force the name of a pointer to be a
557 particular type. This can be used in conjunction with
558 talloc_get_type() to do type checking on void* pointers.
560 It is equivalent to this:
561 talloc_set_name_const(ptr, #type)
563 =-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
564 talloc_get_size(const void *ctx);
566 This function lets you know the amount of memory alloced so far by
567 this context. It does NOT account for subcontext memory.
568 This can be used to calculate the size of an array.