Tcl Source Code

View Ticket
Login
Ticket UUID: 3eace1b21f66da738cd1769e4ec69cb0f50e6029
Title: 'glob -dir $path *' doesn't work for deeply nested paths
Type: Bug Version: 8.5.17
Submitter: john.j.seal Created on: 2015-07-07 17:20:42
Subsystem: 36. Pathname Management Assigned To: nobody
Priority: 5 Medium Severity: Important
Status: Open Last Modified: 2023-08-18 13:19:18
Resolution: None Closed By: nobody
    Closed on:
Description:

The command [glob -dir $path *] fails when $path is very deeply nested; this makes it impossible (AFAICT) to traverse a deeply nested directory hierarchy. It doesn't matter whether $path is a plain path, or begins with the //?/ long path prefix. Some (but not all) of the [file] commands can handle long paths, depending on the Tcl version. Here is a program that illustrates some of the deficiencies and differences:

# Windows [glob] bug demo. For background info see:
# https://groups.google.com/forum/#!topic/comp.lang.tcl/5awD6g40jrg 
# https://core.tcl.tk/tcl/tktview/1479814fffffffffffff
# https://core.tcl.tk/tcl/tktview/3389978fffffffffffff

catch {console show}
puts "Tcl [info patchlevel]"
update

# Make a path >300 characters by adding 5 at a time.
set lvl 0
set path [pwd]
while {[string length $path] <= 300} {
	incr lvl
	append path /path
}
puts "Path length: [string length $path]"

# Test whether [file mkdir] can create a directory under it.
puts -nonewline "File mkdir with plain path"
if {[catch {file mkdir $path/path} err]} {
	puts ": failed"
	puts -nonewline "File mkdir with long path prefix"
	if {[catch {file mkdir //?/$path/path} err]} {
		puts -nonewline ": failed"
		vwait forever
	}
}
puts ""

# Test whether [file exists] can see it.
if {[catch {file exists $path} exists]} {set exists "failed"}
puts "File exists with plain path: $exists"
if {[catch {file exists //?/$path} exists]} {set exists "failed"}
puts "File exists with long path prefix: $exists"

# Test whether [glob] can see it.
if {[catch {llength [glob $path]} matches]} {set matches "failed"}
puts "Glob with plain path: $matches"
if {[catch {llength [glob //?/$path]} matches]} {set matches "failed"}
puts "Glob with long path prefix: $matches"

# Test whether [glob -dir] can see the subdirectory.
if {[catch {llength [glob -dir $path *]} matches]} {set matches "failed"}
puts "Glob -dir with plain path: $matches"
if {[catch {llength [glob -dir //?/$path *]} matches]} {set matches "failed"}
puts "Glob -dir with long path prefix: $matches"

# Test whether [glob] can see the subdirectory.
if {[catch {llength [glob $path/path]} matches]} {set matches "failed"}
puts "Glob on subdir with plain path: $matches"
if {[catch {llength [glob //?/$path/path]} matches]} {set matches "failed"}
puts "Glob on subdir with long path prefix: $matches"

# Test whether [file delete] can delete it.
puts -nonewline "File delete with plain path"
if {[catch {file delete -force [pwd]/path} err]} {
	puts ": failed"
	puts -nonewline "File delete with long path prefix"
	if {[catch {file delete -force //?/[pwd]/path} err]} {
		puts -nonewline ": failed"
		# Delete it iteratively.
		while {$lvl > 0} {
			incr lvl -1
			file rename -force path/path tmp
			file delete -force path 
			file rename -force tmp path
		}
		file delete -force path
	}
}
puts ""
Here are the results for Tcl 8.6.4 and 8.5.17 (note that actually tested this using Tclkits, not full Tcl installs):
Tcl 8.6.4
Path length: 301
File mkdir with plain path: failed
File mkdir with long path prefix
File exists with plain path: 1
File exists with long path prefix: 1
Glob with plain path: 1
Glob with long path prefix: 1
Glob -dir with plain path: failed
Glob -dir with long path prefix: failed
Glob on subdir with plain path: 1
Glob on subdir with long path prefix: 1
File delete with plain path: failed
File delete with long path prefix: failed
Tcl 8.5.17
Path length: 301
File mkdir with plain path: failed
File mkdir with long path prefix
File exists with plain path: 0
File exists with long path prefix: 1
Glob with plain path: failed
Glob with long path prefix: 1
Glob -dir with plain path: failed
Glob -dir with long path prefix: failed
Glob on subdir with plain path: failed
Glob on subdir with long path prefix: 1
File delete with plain path: failed
File delete with long path prefix: failed
As you can see, [glob //?/$path] works on both, but [glob $path] works only on the more recent version of Tcl. But neither [glob -dir //?/$path *] nor [glob -dir $path *] works at all, and that's what you need in order to traverse a directory hierarchy.

User Comments: anonymous added on 2023-08-18 13:19:18:
On windows - you can get deeper down a hierarchy using Twapi find_file_open and find_file_next.
Note that here too - if NTFS long paths haven't been enabled - the //?/ or //./ syntax won't help (probably it doesn't support globs anyway)

The trick with twapi is to use the 'shortname' property of the whole path.
This can be retrieved using the -detail full option with Twapi, or file attributes in Tcl.

This suggests that it should be possible to enhance Tcl's glob, cd etc commands to automatically use the shortname under the hood - but still display the longname.

oehhar added on 2015-07-08 13:43:43:

Great analysis !

Sorry, no solution, no clue, but a wish:

A good starting step for implementation would be to have those tests in the test files [test/fileName.test] (glob) and [test/fileSystem.test] (all others).

Thanks, Harald