kbk	I don't know if it's bytecompilation... have you tried doing the bcc'ed one first?
miguel	ahh ... if I do the bcc'ed first, both behave the same - as the bcc'ed used to do (ie, the wrong one according to the testsuite)
kbk	I think it has to do possibly with losing the state of the shiftjis decoder.
kbk	Any idea what the 'old one found' comment is all about?
miguel	nope
kbk	And what's with [encoding system identity] twice (lines 10 and 23)?  Seems as if that ought to do precisely nothing, since system encoding is already 'identity'
kbk	Hmmm, is this Windows?
kbk	no, never mind, I see it's unix
kbk	The issue is literal lifetime and whether the shiftjis encoding can be found when [ encoding system] is identity.
kbk	3: Stash the current system encoding (to be able to restore it later)
kbk	4: Stash the encoding dirs (to be able to restor later)
kbk	5: Set the system encoding to shiftjis.
kbk	6: Set [encoding dirs] to the path to the wd, converted from whatever the true system encoding is - as if the system encoding were shiftjis. (The path had better be plain ASCII!)
miguel	two notes: (a) I do not grok the encoding stuff at all; (b) I am worried about different behaviour between interpreted and compiled (that should not happen, and I fail to see anything dangerous about it here: why does it?)
kbk	7: Try to convert u+4e4e to shiftjis.  This will load the shiftjis encoding, and may dismiss it again if the 'shiftjis' literal loses its int rep
kbk	Oh, sorry, 7. can't lose the shiftjis encoding 'coz the system encoding is holding a reference to it.
kbk	I still think that the problem is that shiftjis can't be found with [encoding system] and [encoding dirs] set the way they are... that's the true bug.
kbk	But then the bug is masked in one case and not the other by different literal lifetimes, where the encoding is cached in the int rep of the 'shiftjis' literal.
kbk	And this test looks thoroughly bogus - I see no reason to expect it to work with [encoding system] set wrong.
miguel	ahhh ... now I see where you are going
miguel	encoding-11.1 seems to be the same thing
kbk	Might need to replace the 'shiftjis' throughout with something like [join {shift jis} {}] to defeat int rep caching... and then report it as an encoding bug
kbk	(Probably as a bogus test, since using the filesystem with [encoding system] set incorrectly is a horribly, eye-poppingly, face-slappingly, bad thing to do.
kbk	)
miguel	not sure I'd know what to report (yet) - as I am not sure what is supposed to be happening
kbk	Doing so happens to work a good part of the time because most filesystem encodings have ASCII as a proper subset...
kbk	... but that is by no means guaranteed.
miguel	the filesystem usage is easy enough to bypass, right?
kbk	I don't think so; this test seems to be trying to force it to go to the filesystem (perhaps in order to test that it actually does)
miguel	set dirs [list [pwd]]
	encoding system shiftjis ;# incr ref count
	encoding dirs $dirs
kbk	Uhm, the [encoding dirs] is supposed to get rid of cached encodings, and then [encoding convertto] is supposed to be going to the filesystem to load the encoding table all over again.
kbk	I think.
miguel	what I thought should be happening is: as shiftjis is not to be found in [encoding dirs], the thing should fail. So, set [encoding dirs] to anything where the encoding is not to be found. [pwd] is the obvious cheap choice
kbk	And so it could be that encoding *also* has a bug wth its epoch management.
kbk	Ah... I was looking at *your* case, and hadn't pulled up encoding-2.2.
miguel	my case *is* encoding-2.2, copy/pasted
kbk	But I didn't know which of your two results was correct!
miguel	ok
kbk	Hmmm. I don't even know if encoding caches stuff on int reps, without looking.
kbk	But if it does, then it's got a bug with epoch management.
kbk	(And [pwd] misencoded is also a cheap 'not there', so I won't worry too much about all the [encoding system] stuff)
kbk	It appears that [encoding dirs] is changing the encoding path, but the 'shiftjis' encoding, presumably cached on the literal's internal rep, is persisting.
kbk	And it could be that it's simply a matter that this bug has been there for a long time, but that the literal is shared more widely than it was, or doesn't shimmer, or some such.
miguel	11.1 is a bit different; I also get a failure there ... but it might well be a side-effect of the previous one (shiftjis is in the literal for the interp's lifetime)
kbk	Yes, encodings do get cached in an obj's int rep.
kbk	And I don't see anything in tclEncoding.c to spoil the cache when [encoding dirs] changes.
kbk	I think the assumption may be that an encoding is fixed, and a change to [encoding dirs] won't affect any encoding that has been resolved successfully.
kbk	And that's not a terrible assumption, actually...
kbk	... but these tests seem to be trying to make sure that extant encodings are spoilt when the encoding path changes.
miguel	hmmm ... it seems to be relying on Tcl_DecrRefCount to do the spoiling, then? Which is not enough - the thing could be shared (as it is if it is a literal)
kbk	I think that either (a) we decide this is Not A Bug - the 'shiftjis' encoding should be the same everywhere provided that it is found at all, or (b) that we need an epoch number in the cache of encodings (just as we have in so many of our other caches)
kbk	But *that* part is for the maintainer of tclEncoding.c to sort out.
miguel	my take then is that I should ignore this test failure (in the sense that it is not a bug in bc'ing uplevel), and file an encoding bug. Right?
kbk	Right.  I suspect it's just that the bcc has gotten more efficient, and we're now persisting a useful int rep that we previously lost.
kbk	Does it succeed if you replace all the 'shiftjis' literals with [join {shift jis} {}] or some such?
miguel	no failure when using [join {shift jis} {}] ... caching hypothesis confirmed