Ticket UUID: | 582256 | |||
Title: | Releasing Tcl_Obj Allocator | |||
Type: | Patch | Version: | None | |
Submitter: | dkf | Created on: | 2002-07-16 14:24:14 | |
Subsystem: | 41. Memory Allocation | Assigned To: | aku | |
Priority: | 3 Low | Severity: | Minor | |
Status: | Open | Last Modified: | 2016-08-20 16:46:23 | |
Resolution: | None | Closed By: | nobody | |
Closed on: | ||||
Description: |
This patch implements an optional (i.e. selectable through an argument to configure) new object allocator for the Tcl core that has the following properties: * it allocates objects in blocks (similar to the standard Tcl_Obj allocator) * it detects when a block is surplus to requirements (new feature) * it returns the block to the OS when that happens (new feature) The downside to this is that it makes the core slower due to the extra time required to manage a more complex free-list system, and the fact that if a block is discarded and more objects subsequently become necessary then a system call is required to get more memory (though that is of course the whole point.) At time of submission, this code is only verified to work under Solaris 8. | |||
User Comments: |
georgeps added on 2006-01-09 14:19:46:
Logged In: YES user_id=585068 I'm interested in some kind of better Tcl_Obj pool that doesn't use a high-water-mark allocator, because I think it could help performance (as you found). I have profiled Tcl using a modified tcl-bench and gprof in NetBSD x86, and malloc consumes a great deal. The cost of using malloc is probably greater, because it's designed for a wide range of objects. The problem of fragmentation also occurs with malloc and a mixture of Tcl_Obj and other structures, that generally doesn't occur with a specialized pool allocator (although it could influence page release). I am interested in working on a new pooled allocator patch. I have implemented object pools in another project (not thread-safe though), so I think I have enough experience to give it a try, and make sense of your changes. I'm also going to study "Zippy" (the threaded allocator to see what I can learn). :) dkf added on 2006-01-07 16:15:08: Logged In: YES user_id=79902 This code pre-dates Win64 :-) so adapting it to use suitable integer types is perfectly OK with me. The main thing is that there needs to be a way to mask off the lower bits to get the address of the start of the page containing the Tcl_Obj so that the book-keeping structure can be located. The cost of closing and reopening any file descriptor has got to be appreciable because it requires two context switches (plus string parsing to convert the filename into a directory entry, etc.) Some Unix platforms instead do page allocation through mmap()ing a specific invalid handle. Note that the cost of the memset is almost beside the point (it's the update to the page table we really want!) and there's no guarantee that that's what a map of /dev/zero would be doing; taking advantage of the bus width to write more at once would be practical in an OS kernel. In any case, as I wrote, this code is only known to work on Solaris 8, which is pretty old nowadays. If you want to update it to work with newer systems (and a newer Tcl too, including the threaded Tcl_Obj allocator) then be my guest. georgeps added on 2006-01-07 15:14:50: Logged In: YES user_id=585068 This patch will break on some platforms. It assumes that a long can store a pointer value, which is usually true, but isn't required. IIRC Win64 just has 64-bit pointers, and keeps long as a 32-bit type. I also note that you're just supporting mmap. VirtualAlloc can be used in Windows to accomplish the same thing. Specifically this code: PoolBookkeeping *pagePtr = (PoolBookkeeping *) (((long)objPtr) & POOL_ADDR_MASK); This code clearly seems wrong: if (devzero == -123456) { + /* + * This leaks a file-descriptor, but there's no neat way to + * avoid it without opening and closing each time, which is + * desperately inefficient. + */ + devzero = TclOSOpen("/dev/zero", O_RDONLY); + } + if (devzero<0) { + return NULL; + } How much of an impact is it to close and reopen /dev/zero? Have you compared the cost to a memset (gcc builtin)? This code scares me a bit: +# ifndef MAP_FAILED +# define MAP_FAILED ((VOID *)-1) +# endif What platforms lack MAP_FAILED? Can we safely assume that -1 is the right value on such a platform? nwourms added on 2005-02-04 06:07:10: Logged In: YES user_id=556358 Can your please resync with HEAD? dkf added on 2002-07-19 21:49:02: File Deleted - 27230: File Added - 27387: objalloc2.patch dkf added on 2002-07-19 21:49:01: Logged In: YES user_id=79902 Minor fix of stupid fault... dkf added on 2002-07-17 20:43:22: File Deleted - 27126: File Added - 27230: objalloc2.patch Logged In: YES user_id=79902 Experience shows that the allocator that this provides significantly speeds up tclbench on Solaris8, with slightly better overall gains when the page-size (modified through changing the Makefile) is 32kB than when it is the default (8kB). This is not something I really anticipated! dkf added on 2002-07-16 21:44:00: File Added - 27126: objalloc2.patch dkf added on 2002-07-16 21:43:59: Logged In: YES user_id=79902 D'oh! Try again... |
Attachments:
- objalloc2.patch [download] added by dkf on 2002-07-19 21:49:01. [details]