Tcl Source Code

On a Windows machine accessing a Linux remote PC directory shared with Samba, in wish with Tcl/Tk 8.4.14 binary distribution from ActiveState (g:/ is the samba drive):

% file exists g:/
1
% file readable g:/
0
% dir g:
[shows the network drive content]

This is also the result for all directories in the hierarchy. The directories exist, they are not readable (BUG ?!?), but one can change to the directories with "cd" and the "dir" command shows their content.

Now on a file:

% file exists g:/<hierarchy>/pll_noise.sci
1
% file readable g:/<hierarchy>/pll_noise.sci
0
% file owned g:/<hierarchy>/pll_noise.sci
1
% file size g:/<hierarchy>/pll_noise.sci
8338

This file, pll_noise.sci was created by an admin user.

The OS is a german Win2000 SP4 (build 5.00.2195). The samba network drive is only accessed from Windows, the admin user has full rights to the complete network drive (the hierarchy has been created from Windows).

There are only 7-bit ASCII-characters in the network path, no white space, etc. The total length of the Windows network path including computername and all slashes is 62 characters and is of the structure \\\computer\dir1\dir2\dir3\dir4\dir5\dir6\pll_noise.sci

The drive is used by several other Windows programs without problems (MSOffice, OpenOffice, Acroread). The file access properties of pll_noise.sci show all "allow" checkboxes checked. The directories of the hierarchy have no checks in neither "allow" nor "disallow" checkboxes, the checkbox "inherit properties from parent directory" is also blank, but in the "advanced" properties the user is granted full access.

The file pll_noise.sci can be read with OpenOffice2, Notepad, etc, through the Samba network. But the check on file readable fails and this sounds wrong to me.

Other files on the network drive (regardless of their position in the hierarchy) also have file readable to return 0 (wrong!), but can be opened with other Windows programs.

Moving the file to a shared WinXP folder on an NTFS partition on another PC lets file readable return 1 correctly. I guess that the samba share is the cause here for an incorrect result of file readable.

Francois

Oh, right.
Thanks, Don.

After repair of one test, things test OK for me.

Substitution to produce the value for the -result option of
test winFCmd-12.6.2 happens always and everywhere.

Sounds like it's just a testing error then?

how it can occure by -constraints {win}?

I object! :)

Change committed to core-8-5-branch breaks the test suite on Centos Linux 7:

winFCmd.test
Test file error: no such variable
    (read trace on "::env(TEMP)")
    invoked from within
"file normalize $::env(TEMP)"
    invoked from within
"string tolower [file normalize $::env(TEMP)]/td1"
    invoked from within
"test winFCmd-12.6.2 {ConvertFileNameFormat: absolute path with drive (in temp folder)} -setup {
    catch {file delete -force -- $::env(TEMP)/td1}
} -..."
    (file "/local/tmp/dgp/fossil/tcl8.5/tests/winFCmd.test" line 911)

So no objections until now, then I'll merge it now.

Thx, Harald!

Thus I will wait few days for some possible objections and then 
it will be merged for current branches >= 8.5.

Sergey,

the test is done identically to my post in this ticket dated "2017-06-15 14:41:42".

The only difference is, that the bug is corrected:

% file readable u:/test/test.txt
1
% file exists u:/test/test.txt
1
% set h [open u:/test/test.txt]
file 580e58

Congratulations, Harald

Yes, of cause. Test result probably 2018-04-17 (next Tuesday).

Thanks, Harald

@Harald Could you test it on your system?

So I think I fixed it now (see [0bce348e6b], branch [fix-1613456fff], over 8.5 because all versions are affected ATM).

This has fast inplace check of security permissions using CreateFile (without modification of access time) and looks like more faster and stable solution without direct check of security permissions by optimal terms;

So it checks positive/negative case of file readable, writable and executable.

In-between I tried another one using GetNamedSecurityInfo/GetEffectiveRightsFromAcl pair with CURRENT_USER trustee... It had the same behaviour and was still slower as current implementation.

As bonus the current performance progress:

%  file readable $readable_path ;# positive case
1
- 3882.55 µs/# 65 # 257.56 #/sec 252.366 nett-ms
+ 80.7825 µs/# 3094 # 12378.9 #/sec 249.941 nett-ms

%  file readable $noreadright_path ;# negative case (explicit permissions)
0
- 5130.85 µs/# 195 # 194.90 #/sec 1000.515 nett-ms
+ 71.6337 µs/# 3489 # 13959.9 #/sec 249.930 nett-ms

Almost the same time for writable/executable, if running into permissions check (not readonly/extentions check, there are still faster in both cases).

If accepted, I'll merge/reintegrate it in all current branches >= 8.6.

Attached access.dll, uses this code, which does not match my prior test:

#include <tcl.h>
#include <string.h>
#include <windows.h>
#include <io.h>

int accessObjCmd (
  ClientData cd,
  Tcl_Interp* interp,
  int objc,
  Tcl_Obj * const objv[]
  )
{
  int rc, lt1;
  const char* s1;

  if (objc != 2) {
    Tcl_WrongNumArgs(interp, 1, objv, "filename");
    return TCL_ERROR;
  }
  s1 = Tcl_GetStringFromObj(objv[1], &lt1);
  rc = _access_s (s1, R_OK);
  Tcl_SetObjResult (interp, Tcl_NewBooleanObj (1-rc));
  return TCL_OK;
}

int Access_Init (Tcl_Interp *interp)
{
  if (!Tcl_InitStubs (interp,"8.3",0)) {
    return TCL_ERROR;
  }
  Tcl_CreateObjCommand (interp,"access", accessObjCmd, NULL, NULL);
  Tcl_PkgProvide (interp,"access","0.1");

  return TCL_OK;
}

Sergey,

I am sorry. I will be back in the office 9th of April. No test before this date from me.

I don't have ffidl but may try with bawt or Ashok BI which might have it.

Sorry, Harald

It almost seems better to use the open/close check, even though that would modify the access time.  Then the microsoft internals would handle the ACLs, the offline file api, network shares, symlinks, etc.

It will be slow, but it will always work.

The file access time could be reset, but that's even slower.

Arghhh, who can read has a clear advantage - so RTFM and it (msdn-docu) says about "_access_s":

"This function only checks whether the file and directory are read-only or not, it does not check the filesystem security settings."

Thus unfortunately no go without GetNamedSecurityInfo or similar...

Great, it works! At least for your case (virtualbox)...

The only question would be: the same on samba shares (original issue)?

@Harald, @François and others who can reproduce it: please try access_s variant (ffidl- resp. C-attempt of Brad)...

sebres added on 2018-03-22 10:22:33:

> I don't think so:
>  1. Note, that `_access_s` returns 0 if the file has the given mode. 

Oh, then the results are correct and _access_s is working, whereas
the current implementation of 'file readable' does not.

I only looked at _access_s in the header file and did not read the 
documentation for it.  My fault.

>  2. Don't use Tcl_GetStringFromObj on joined path ("/" vs "\\"), rather use
     `Tcl_FSGetNativePath(objv[1])` or `file nativename` in script-level before
     you supply it to your access-command.

Not necessary.  All windows internals will handle / correctly.
That's only a command line thing.  But I had tried that just in
case and the results were the same.

> Well, I wasn't expecting that result.
> If the virtualbox network share is buggy, there's nothing we can do about that.

I don't think so:

  1. Note, that `_access_s` returns 0 if the file has the given mode. 
  2. Don't use Tcl_GetStringFromObj on joined path ("/" vs "\\"), rather use
     `Tcl_FSGetNativePath(objv[1])` or `file nativename` in script-level before
     you supply it to your access-command.

So try again, please...

C Code:

#include <tcl.h>
#include <string.h>
#include <windows.h>
#include <io.h>

int accessObjCmd (
  ClientData cd,
  Tcl_Interp* interp,
  int objc,
  Tcl_Obj * const objv[]
  )
{
  int rc, lt1;
  const char* s1;

  if (objc != 2) {
    Tcl_WrongNumArgs(interp, 1, objv, "filename");
    return TCL_ERROR;
  }
  s1 = Tcl_GetStringFromObj(objv[1], &lt1);
  rc = _access_s (s1, R_OK);
  Tcl_SetObjResult (interp, Tcl_NewIntObj (rc));
  return TCL_OK;
}

int Access_Init (Tcl_Interp *interp)
{
  if (!Tcl_InitStubs (interp,"8.3",0)) {
    return TCL_ERROR;
  }
  Tcl_CreateObjCommand (interp,"access", accessObjCmd, NULL, NULL);
  Tcl_PkgProvide (interp,"access","0.1");

  return TCL_OK;
}


Base test script (Tcl 8.6.8):

try {
  load [pwd]/access[info sharedlibextension]
} on error {err res} {
  puts $res
}
if { [info commands access] ne "access" } {
  exit 1
}

set ns [lindex $::argv 0]

set dn [file join $ns testdir2]
set fn [file join $dn xyzzy]
file mkdir $dn
set fh [open $fn w]
close $fh

puts "$dn [file readable $dn]"
puts "$fn [file readable $fn]"
puts "$dn [access $dn]"
puts "$fn [access $fn]"

C:\Users\bll\Desktop\BallroomDJ\test.dir>..\windows\64\tcl\bin\tclsh.exe t.tcl F
:/
F:/testdir2 0
F:/testdir2/xyzzy 0
F:/testdir2 0
F:/testdir2/xyzzy 0

C:\Users\bll\Desktop\BallroomDJ\test.dir>..\windows\64\tcl\bin\tclsh.exe t.tcl C
:/
C:/testdir2 1
C:/testdir2/xyzzy 1
C:/testdir2 0
C:/testdir2/xyzzy 0

Well, I wasn't expecting that result.
If the virtualbox network share is buggy, there's nothing we can do about that.

I don't have virtualbox and with vmware shares it works fine.
If you could still reproduce it (and can use ffidl), could you please compare _access_s with readable as suggested below?

I had run into this issue trying to run tcltest using the files on a virtualbox network share.  In my case, it was the directories that would return false
on a 'file readable'.

https://groups.google.com/d/msg/comp.lang.tcl/jCPoMiftl7U/Hqfu8jt2DAAJ

Admittedly I still could not reproduce it directly, but I saw in some logs of my customers the signals it may indeed take place (successful opening implicitly after logging it was not readable).

Looking into NativeAccess in tclWinFile.c could possibly explain it, because IMHO current realization may be not always suitable for some constellations with samba shares. Additionally it could cause the race conditions inside.

By the way, I believe it should be extremely inperformant on some network shares, so I had tested it and indeed noticed a high performance degradation. Even open/close is more faster.

So I have bound (via ffidl) the CRT-access_s function (just to compare it also), below you'll find the results of:

package require Ffidlrt
ffidl::callout _access_s {pointer-utf8 int} int [ffidl::symbol msvcr90.dll _access_s] stdcall
ffidl::callout _waccess_s {pointer int} int [ffidl::symbol msvcr90.dll _waccess_s] stdcall

set path {z:\any\path\with\read\access\on\network\drive.txt}

# timerate it 250ms ... #

%  file exists $path 
58.0534 µs/# 4305 # 17225.5 #/sec 249.920 nett-ms

%  set f [open $path rb]; close $f 
145.348 µs/# 1721 # 6880.0 #/sec 250.144 nett-ms

%  file readable $path 
3882.55 µs/# 65 # 257.56 #/sec 252.366 nett-ms

%  _access_s $path 04 
54.0879 µs/# 4620 # 18488.4 #/sec 249.886 nett-ms

%  _waccess_s [::ffidl::get-unicode $path] 04 
55.5223 µs/# 4501 # 18010.8 #/sec 249.906 nett-ms

@Harald, possibly if you can reproduce it again, and if you've ffidl, could you possibly test this `expr {[_access_s $path 04] == 0}` will return 1 (GRANTED), where `file readable $path` still returns 0.

In this case we could rewrite it using this CRT-primitives (and for exec/owner check using `fstat` with _S_IREAD|_S_IWRITE|_S_IEXEC combinations or something similar).

I have also found that [file readable|writable] returns 0 on Windows when it should return 1 - in this case for the pseudo-network shared directory of VirtualBox when Windows runs in a VM.

The shared directory on the (Linux) host is represented as \\vboxsrv and also as drive "E:". This is a wish 8.6.6 console session:

(bin) 1 % cd E:/
() 2 % file mkdir temp1
() 3 % file writable temp1
0
() 4 % file mkdir temp1/temp2
() 5 % set fout [open temp1/temp.txt w]
file3815c60
() 6 % puts $fout Hi!
() 7 % close $fout
() 8 % file writable temp1
0
() 9 % 
() 9 % 
() 9 % file readable temp1/temp.txt
0
() 10 % set fin [open temp1/temp.txt]
file37da530
() 11 % read $fin
Hi!

() 12 % close $fin
() 13 % file readable temp1/temp.txt
0
() 14 % 

The shared directory on Linux is owned by the same user that runs VirtualBox. Files are created with conventional permissions: 775 for directories, 664 for files. Changing these to 777, 666 makes no difference to [file readable|writable] in Tcl in the Windows guest OS, even in a fresh Tcl interpreter.

Jeff mentioned below a similar bug in Tcl when using VMware - [1661378fffffffff].

Sorry, I meant [512a5af3947e9bb3].

I also ran the test suite, if someone interested.

I also verified, that the bug does not appear on the same samba share with my normal Laptop where the password of the server and the login are identical and thus, the procedure can not be followed.

Thank you, Harald

> and a tclsh "core-8-6-branch" [6203735037] with compiled by me with ms-vc6++

Just to be sure: I meant current "core-8-6-branch", not the [6203735037]...
The commit [6203735037] was just mentioned for the similar problem (that was reproducible on some samba-shares only).

Sergey,

thank you for the post and the test. This is highly appreciated.

I can still reproduce the issue with the following test with tcl 8.6.3 and current core-8-6-branch.

How to reproduce:

• Login with different password than network share so windows can not automatically connect
• start tclsh and do:

% file readable u:/test/test.txt
0
% file exists u:/test/test.txt
0
% set h [open u:/test/test.txt r]
could not open "u:/test/test.txt": no such file or directory

• Open windows explorer and click on another network drive of the same server (S:) -> Server Login window is shown and credentials are entered
• Enter in tclsh:

% file readable u:/test/test.txt
0
% file exists u:/test/test.txt
0
% set h [open u:/test/test.txt r]
could not open "u:/test/test.txt": no such file or directory

• In windows explorer click on drive letter "U" to list U:\
• Enter in tclsh:

% file readable u:/test/test.txt
0
% file exists u:/test/test.txt
1
% set h [open u:/test/test.txt]
file 580e58

This is Windows Vista 32bit with CentOS 6.8 Samba. It is once a wish 8.6.3 and a tclsh "core-8-6-branch" [6203735037] with compiled by me with ms-vc6++

I cannot reproduce it on current 8.5, 8.6 and trunk (winXP, win7, win2K8).

Possibly it goes to typos-fix (in winPipe) that I made a time ago or it was something else (e. g. I saw several commits round about native/normalized path, etc).

Harald, can you please test it again with current core-8-6-branch with your samba share?
Just to be sure it is not a bug with some special share-case resp. version, like unmapped users samba-bug [6203735037].

I reopened this bug as I am still viewing this issue with TCL8.6.3.

I am with Windows 8 64 bit on a samba share. The file is a text file written by me.

% file exists u:/test/test.txt
1
% file readable u:/test/test.txt
0
% set h [open u:/test/test.txt r]
file388b540
% close $h

When I copy the file with windows explorer to C:, I get the warning "Eventual risky file", which is true for all files on the share. This might eventually be a clue for the behavior.

Is this still an issue? (I have completely the wrong platform to check myself…)

I jusr ran into this problem with Tcl 8.5.13! The file (tkcon.tcl) is owned by the user and can be read (aka source ....../tkcon.tcl) works fine, but 'file readable' reports 0, but 'file exists' and 'file owned' both return 1 and the owner is the current user.

Patch from fix-win-native-access branch
committed for 8.4.20, 8.5.12, and 8.6b3.

Continued testing strongly encouraged.

In particular, the #define UNICODE variant
of the trunk is likely to have some lingering
problems.

Well, five years at my report, I had to build a specific configuration with two conputers specially to test this patch.

The results are apparently good: file readable answers 1 on the samba shared directory and on the file located in this share as well.

The tricky thing is that fossil updating to just before the patch does not allow me to reproduce the original issue. Reasons are probably numerous: not the same computers, different linux distro, samba and windows versions, samba configuration file lost since my initial report, can't remember the exact configuration...

Sorry for not being very informative, but I hope you understand that I had worked around the issue in my application right when it happened, and despite I kept the stuff alive for some time I saw no reason to maintain all this for an undefined amount of time that turned out to be five years.

Well, at least the patch does not seem to harm...

I may try further to reproduce the original issue, but no promises here, sorry.
BtW, where did the patch come from? Did it come with a well-defined test case along with a well-defined (samba) configuration that I could reproduce?

The branch 'fix-win-native-access' includes a patch that
may fix the bug reported here.

  http://core.tcl.tk/tcl/info/0aabf0529e

I'm not equipped to test the effectiveness of the
patch, or whether it introduces any new test
failures or other problems.  I'd be grateful if folks
who can and who are intersted in this bug and related
bugs could do so, and report back whether the patch
is suitable for merging into the release branches.

Logged In: YES 
user_id=1245417
Originator: YES

Hi,

Yes, still here with cvs HEAD version (info pa spits 8.6a0).

Francois

Logged In: YES 
user_id=1245417
Originator: YES

I will try if this specific bug is still here in 8.5.2, please allow me for some days to rebuild the configuration I had when I reported this more than one year ago.

In the meantime, see a similar report in comp.lang.tcl:

http://groups.google.com/group/comp.lang.tcl/browse_thread/thread/18ec9547f7700607?hl=en#

Francois

Logged In: YES 
user_id=80530
Originator: NO


Is this bug present in Tcl 8.5.2 ?

Logged In: YES 
user_id=72656
Originator: NO

related to 1661378 for vmware shared folders?

Logged In: YES 
user_id=598324
Originator: NO

I can confirm that I have this problem with samba-3.0.24,1 on FreeBSD-6.x 

From a windows2000 machine 'file writable' may declare a perfectly writable path unwritable. 

%file writable //machine/share
0
file mkdir //machine/share/xxx
%file exists //machine/share/xxx
1

Logged In: YES 
user_id=1245417
Originator: YES

Hi,

I tracked the problem to the following narrower issue.

In some Linux directory shared with Samba, create any file myfile.txt.

Then play with the access rigths and see the results:

chmod 0600 myfile.txt
  Then file readable $thefile answers:
     1 in wish run on the Linux machine
     0 in wish run on the Windows machine
  And file writable $thefile answers:
     1 in wish run on the Linux machine
     0 in wish run on the Windows machine

chmod 0606 myfile.txt
  Then file readable $thefile answers:
     1 in wish run on the Linux machine
     1 in wish run on the Windows machine
  And file writable $thefile answers:
     1 in wish run on the Linux machine
     1 in wish run on the Windows machine

i.e. the file must be granted access rights for the "Others" users group, otherwise it is neither readable nor writable, even by the user who created it.

The smb.conf has
  map to guest = never
and I created a Linux user with the same name and password as the Windows user, so that there is no login dialog when the Windows machine connects to the Linux samba share. The Windows login and passwords are sent by Windows to Samba, and Samba has this user included in its smbpasswd file.


To sum up:

  It seems that a file once readable and writable by some user on Linux is not readable nor writable later by this same user if this user connects through the samba share.

  When run through a Samba share, file readable/writable appears to check not only the user rights but also the "others" access rights, and this is odd.

Francois