Tcl Source Code

2012-09-07  Harald Oehlmann  <[email protected]>

	IMPLEMENTATION OF TIP#404.

	* library/msgcat/msgcat.tcl:	[FRQ 3544988]: (Backport from Tcl 8.6)
	* library/msgcat/pkgIndex.tcl:	New commands [mcflset] and [mcflmset]
	* unix/Makefile.in:		to set mc entries with implicit message
	* win/Makefile.in:		file locale. Bump to 1.5.0.
	* tests/msgcat.test:

2012-09-07  Alexandre Ferrieux <[email protected]>

	* unix/tclUnixNotfy.c Backport of 2008-12-12 8.6 commit: Fix
	missing CLOEXEC on internal pipes [2417695]

2012-08-25  Donal K. Fellows  <[email protected]>

=> registry 1.2.2

2012-07-16 (bug fix)[3544683] reentrant syscalls on BSD (cassoff,fellows)

2012-07-24 (bug fix) stop mem corruption in stacked channel events (max,porter)

2012-07-25 (bug fix)[3546275] [auto_execok] search match [exec] (danckaert)

2012-09-12 (TIP 404) New msgcat commands [mcflset], [mcflmset] (oehlmann)
=> msgcat 1.5.0

Many revisions to better support a Cygwin environment (nijtmans)

--- Released 8.5.12, July 27, 2011 --- See ChangeLog for details ---

.BS
'\" Note:  do not modify the .SH NAME line immediately below!
.SH NAME
msgcat \- Tcl message catalog
.SH SYNOPSIS
\fBpackage require Tcl 8.5\fR
.sp
\fBpackage require msgcat 1.5.0\fR
.sp
\fB::msgcat::mc \fIsrc-string\fR ?\fIarg arg ...\fR?
.sp
\fB::msgcat::mcmax ?\fIsrc-string src-string ...\fR?
.sp
\fB::msgcat::mclocale \fR?\fInewLocale\fR?
.sp
\fB::msgcat::mcpreferences\fR
.sp
\fB::msgcat::mcload \fIdirname\fR
.sp
\fB::msgcat::mcset \fIlocale src-string \fR?\fItranslate-string\fR?
.sp
\fB::msgcat::mcmset \fIlocale src-trans-list\fR
.sp
.VS "TIP 404"
\fB::msgcat::mcflset \fIsrc-string \fR?\fItranslate-string\fR?
.sp
\fB::msgcat::mcflmset \fIsrc-trans-list\fR
.VE "TIP 404"
.sp
\fB::msgcat::mcunknown \fIlocale src-string\fR
.BE
.SH DESCRIPTION
.PP
The \fBmsgcat\fR package provides a set of functions
that can be used to manage multi-lingual user interfaces.

.PP
Use of the message catalog is optional by any application
or package, but is encouraged if the application or package
wishes to be enabled for multi-lingual applications.
.SH COMMANDS
.TP
\fB::msgcat::mc \fIsrc-string\fR ?\fIarg arg ...\fR?
.
Returns a translation of \fIsrc-string\fR according to the
user's current locale.  If additional arguments past \fIsrc-string\fR
are given, the \fBformat\fR command is used to substitute the
additional arguments in the translation of \fIsrc-string\fR.
.RS
.PP
\fB::msgcat::mc\fR will search the messages defined

application can pass the English string through \fB::msgcat::mc\fR and
use the result.  If an application is written for a single language in
this fashion, then it is easy to add support for additional languages
later simply by defining new message catalog entries.
.RE
.TP
\fB::msgcat::mcmax ?\fIsrc-string src-string ...\fR?
.
Given several source strings, \fB::msgcat::mcmax\fR returns the length
of the longest translated string.  This is useful when designing
localized GUIs, which may require that all buttons, for example, be a
fixed width (which will be the width of the widest button).
.TP
\fB::msgcat::mclocale \fR?\fInewLocale\fR?
.
This function sets the locale to \fInewLocale\fR.  If \fInewLocale\fR
is omitted, the current locale is returned, otherwise the current locale
is set to \fInewLocale\fR.  msgcat stores and compares the locale in a
case-insensitive manner, and returns locales in lowercase.
The initial locale is determined by the locale specified in
the user's environment.  See \fBLOCALE SPECIFICATION\fR
below for a description of the locale string format.
.TP
\fB::msgcat::mcpreferences\fR
.
Returns an ordered list of the locales preferred by
the user, based on the user's language specification.
The list is ordered from most specific to least
preference.  The list is derived from the current
locale set in msgcat by \fB::msgcat::mclocale\fR, and
cannot be set independently.  For example, if the
current locale is en_US_funky, then \fB::msgcat::mcpreferences\fR

returns \fB{en_US_funky en_US en {}}\fR.

.TP
\fB::msgcat::mcload \fIdirname\fR
.
Searches the specified directory for files that match
the language specifications returned by \fB::msgcat::mcpreferences\fR
(note that these are all lowercase), extended by the file extension
.QW .msg .
Each matching file is
read in order, assuming a UTF-8 encoding.  The file contents are
then evaluated as a Tcl script.  This means that Unicode characters
may be present in the message file either directly in their UTF-8
encoded form, or by use of the backslash-u quoting recognized by Tcl
evaluation.  The number of message files which matched the specification
and were loaded is returned.
.TP
\fB::msgcat::mcset \fIlocale src-string \fR?\fItranslate-string\fR?
.
Sets the translation for \fIsrc-string\fR to \fItranslate-string\fR
in the specified \fIlocale\fR and the current namespace.  If
\fItranslate-string\fR is not specified, \fIsrc-string\fR is used
for both.  The function returns \fItranslate-string\fR.
.TP
\fB::msgcat::mcmset \fIlocale src-trans-list\fR
.
Sets the translation for multiple source strings in
\fIsrc-trans-list\fR in the specified \fIlocale\fR and the current
namespace.
\fIsrc-trans-list\fR must have an even number of elements and is in
the form {\fIsrc-string translate-string\fR ?\fIsrc-string
translate-string ...\fR?} \fB::msgcat::mcmset\fR can be significantly
faster than multiple invocations of \fB::msgcat::mcset\fR. The function
returns the number of translations set.
.TP
\fB::msgcat::mcflset \fIsrc-string \fR?\fItranslate-string\fR?
.VS "TIP 404"
Sets the translation for \fIsrc-string\fR to \fItranslate-string\fR in the
current namespace for the locale implied by the name of the message catalog
being loaded via \fB::msgcat::mcload\fR.  If \fItranslate-string\fR is not
specified, \fIsrc-string\fR is used for both.  The function returns
\fItranslate-string\fR.
.VE "TIP 404"
.TP
\fB::msgcat::mcflmset \fIsrc-trans-list\fR
.VS "TIP 404"
Sets the translation for multiple source strings in \fIsrc-trans-list\fR in
the current namespace for the locale implied by the name of the message
catalog being loaded via \fB::msgcat::mcload\fR. \fIsrc-trans-list\fR must
have an even number of elements and is in the form {\fIsrc-string
translate-string\fR ?\fIsrc-string translate-string ...\fR?}
\fB::msgcat::mcflmset\fR can be significantly faster than multiple invocations
of \fB::msgcat::mcflset\fR. The function returns the number of translations set.
.VE "TIP 404"
.TP
\fB::msgcat::mcunknown \fIlocale src-string\fR
.
This routine is called by \fB::msgcat::mc\fR in the case when
a translation for \fIsrc-string\fR is not defined in the
current locale.  The default action is to return
\fIsrc-string\fR.  This procedure can be redefined by the
application, for example to log error messages for each unknown
string.  The \fB::msgcat::mcunknown\fR procedure is invoked at the
same stack context as the call to \fB::msgcat::mc\fR.  The return value

specifies U.S. English.
.PP
When the msgcat package is first loaded, the locale is initialized
according to the user's environment.  The variables \fBenv(LC_ALL)\fR,
\fBenv(LC_MESSAGES)\fR, and \fBenv(LANG)\fR are examined in order.
The first of them to have a non-empty value is used to determine the
initial locale.  The value is parsed according to the XPG4 pattern
.PP
.CS
language[_country][.codeset][@modifier]
.CE
.PP
to extract its parts.  The initial locale is then set by calling
\fB::msgcat::mclocale\fR with the argument 
.PP
.CS
language[_country][_modifier]
.CE
.PP
On Windows and Cygwin, if none of those environment variables is set,
msgcat will attempt to extract locale information from the registry.
From Windows Vista on, the RFC4747 locale name "lang-script-country-options"
is transformed to the locale as "lang_country_script" (Example:
sr-Latn-CS -> sr_cs_latin). For Windows XP, the language id is
transformed analoguously (Example: 0c1a -> sr_yu_cyrillic).
If all these attempts to discover an initial locale from the user's
environment fail, msgcat defaults to an initial locale of
.QW C .
.PP
When a locale is specified by the user, a
.QW "best match"
search is performed during string translation.  For example, if a user
specifies

en_GB_Funky, the locales
.QW en_GB_Funky ,
.QW en_GB ,
.QW en
and
.MT
(the empty string)

are searched in order until a matching translation
string is found.  If no translation string is available, then
\fB::msgcat::mcunknown\fR is called.
.SH "NAMESPACES AND MESSAGE CATALOGS"
.PP
Strings stored in the message catalog are stored relative
to the namespace from which they were added.  This allows
multiple packages to use the same strings without fear
of collisions with other packages.  It also allows the
source string to be shorter and less prone to typographical
error.
.PP
For example, executing the code
.PP
.CS
\fB::msgcat::mcset\fR en hello "hello from ::"
namespace eval foo {
    \fB::msgcat::mcset\fR en hello "hello from ::foo"
}
puts [\fB::msgcat::mc\fR hello]
namespace eval foo {puts [\fB::msgcat::mc\fR hello]}
.CE
.PP
will print
.PP
.CS
hello from ::
hello from ::foo
.CE
.PP
When searching for a translation of a message, the
message catalog will search first the current namespace,
then the parent of the current namespace, and so on until
the global namespace is reached.  This allows child namespaces to
.QW inherit
messages from their parent namespace.
.PP
For example, executing (in the
.QW en
locale) the code
.PP
.CS
\fB::msgcat::mcset\fR en m1 ":: message1"
\fB::msgcat::mcset\fR en m2 ":: message2"
\fB::msgcat::mcset\fR en m3 ":: message3"
namespace eval ::foo {
    \fB::msgcat::mcset\fR en m2 "::foo message2"
    \fB::msgcat::mcset\fR en m3 "::foo message3"
}
namespace eval ::foo::bar {
    \fB::msgcat::mcset\fR en m3 "::foo::bar message3"
}
namespace import \fB::msgcat::mc\fR
puts "[\fBmc\fR m1]; [\fBmc\fR m2]; [\fBmc\fR m3]"
namespace eval ::foo {puts "[\fBmc\fR m1]; [\fBmc\fR m2]; [\fBmc\fR m3]"}
namespace eval ::foo::bar {puts "[\fBmc\fR m1]; [\fBmc\fR m2]; [\fBmc\fR m3]"}
.CE
.PP
will print
.PP
.CS
:: message1; :: message2; :: message3
:: message1; ::foo message2; ::foo message3
:: message1; ::foo message2; ::foo::bar message3
.CE
.SH "LOCATION AND FORMAT OF MESSAGE FILES"
.PP
Message files can be located in any directory, subject
to the following conditions:
.IP [1]
All message files for a package are in the same directory.
.IP [2]
The message file name is a msgcat locale specifier (all lowercase) followed by
.QW .msg .
For example:
.PP
.CS
es.msg    \(em spanish
en_gb.msg \(em United Kingdom English
.CE
.PP
\fIException:\fR The message file for the root locale
.MT
is called
.QW \fBROOT.msg\fR .
This exception is made so as not to
cause peculiar behavior, such as marking the message file as
.QW hidden
on Unix file systems.

.IP [3]
The file contains a series of calls to \fBmcflset\fR and
\fBmcflmset\fR, setting the necessary translation strings
for the language, likely enclosed in a \fBnamespace eval\fR
so that all source strings are tied to the namespace of
the package. For example, a short \fBes.msg\fR might contain:
.PP
.CS
namespace eval ::mypackage {
    \fB::msgcat::mcflset\fR "Free Beer!" "Cerveza Gracias!"
}
.CE
.SH "RECOMMENDED MESSAGE SETUP FOR PACKAGES"
.PP
If a package is installed into a subdirectory of the
\fBtcl_pkgPath\fR and loaded via \fBpackage require\fR, the
following procedure is recommended.
.IP [1]
During package installation, create a subdirectory
\fBmsgs\fR under your package directory.
.IP [2]
Copy your *.msg files into that directory.
.IP [3]
Add the following command to your package initialization script:

.PP
.CS
# load language files, stored in msgs subdirectory
\fB::msgcat::mcload\fR [file join [file dirname [info script]] msgs]
.CE
.SH "POSITIONAL CODES FOR FORMAT AND SCAN COMMANDS"
.PP
It is possible that a message string used as an argument
to \fBformat\fR might have positionally dependent parameters that
might need to be repositioned.  For example, it might be
syntactically desirable to rearrange the sentence structure
while translating.
.PP
.CS
format "We produced %d units in location %s" $num $city
format "In location %s we produced %d units" $city $num
.CE
.PP
This can be handled by using the positional
parameters:
.PP
.CS
format "We produced %1\e$d units in location %2\e$s" $num $city
format "In location %2\e$s we produced %1\e$d units" $num $city
.CE
.PP
Similarly, positional parameters can be used with \fBscan\fR to
extract values from internationalized strings. Note that it is not
necessary to pass the output of \fB::msgcat::mc\fR to \fBformat\fR
directly; by passing the values to substitute in as arguments, the
formatting substitution is done directly.
.PP
.CS
\fBmsgcat::mc\fR {Produced %1$d at %2$s} $num $city
# ... where that key is mapped to one of the
# human-oriented versions by \fBmsgcat::mcset\fR
.CE
.SH CREDITS
.PP
The message catalog code was developed by Mark Harrison.
.SH "SEE ALSO"
format(n), scan(n), namespace(n), package(n)
.SH KEYWORDS
internationalization, i18n, localization, l10n, message, text, translation
.\" Local Variables:
.\" mode: nroff
.\" End:

#
# See the file "license.terms" for information on usage and redistribution
# of this file, and for a DISCLAIMER OF ALL WARRANTIES.

package require Tcl 8.5
# When the version number changes, be sure to update the pkgIndex.tcl file,
# and the installation directory in the Makefiles.
package provide msgcat 1.5.0

namespace eval msgcat {
    namespace export mc mcload mclocale mcmax mcmset mcpreferences mcset \
	    mcunknown mcflset mcflmset

    # Records the current locale as passed to mclocale
    variable Locale ""

    # Records the list of locales to search
    variable Loclist {}

    # Records the locale of the currently sourced message catalogue file
    variable FileLocale

    # Records the mapping between source strings and translated strings.  The
    # dict key is of the form "<locale> <namespace> <src>", where locale and
    # namespace should be themselves dict values and the value is
    # the translated string.
    variable Msgs [dict create]

# Arguments:
#	langdir		The directory to search.
#
# Results:
#	Returns the number of message catalogs that were loaded.

proc msgcat::mcload {langdir} {
    variable FileLocale
    # Save the file locale if we are recursively called
    if {[info exists FileLocale]} {
	set nestedFileLocale $FileLocale
    }
    set x 0
    foreach p [mcpreferences] {
	if { $p eq {} } {
	    set p ROOT
	}
	set langfile [file join $langdir $p.msg]
	if {[file exists $langfile]} {
	    incr x
	    set FileLocale [string tolower [file tail [file rootname $langfile]]]
	    if {"root" eq $FileLocale} {
		set FileLocale ""
	    }
	    uplevel 1 [list ::source -encoding utf-8 $langfile]
	    unset FileLocale
	}
    }
    if {[info exists nestedFileLocale]} {
	set FileLocale $nestedFileLocale
    }
    return $x
}

# msgcat::mcset --
#
#	Set the translation for a given string in a specified locale.

    set ns [uplevel 1 [list ::namespace current]]

    set locale [string tolower $locale]

    dict set Msgs $locale $ns $src $dest
    return $dest
}

# msgcat::mcflset --
#
#	Set the translation for a given string in the current file locale.
#
# Arguments:
#	src		The source string.
#	dest		(Optional) The translated string.  If omitted,
#			the source string is used.
#
# Results:
#	Returns the new locale.

proc msgcat::mcflset {src {dest ""}} {
    variable FileLocale
    variable Msgs

    if {![info exists FileLocale]} {
	return -code error \
	    "must only be used inside a message catalog loaded with ::msgcat::mcload"
    }
    if {[llength [info level 0]] == 2} { ;# dest not specified
	set dest $src
    }

    set ns [uplevel 1 [list ::namespace current]]
    dict set Msgs $FileLocale $ns $src $dest
    return $dest
}

# msgcat::mcmset --
#
#	Set the translation for multiple strings in a specified locale.
#
# Arguments:
#	locale		The locale to use.

    set locale [string tolower $locale]
    set ns [uplevel 1 [list ::namespace current]]

    foreach {src dest} $pairs {
	dict set Msgs $locale $ns $src $dest
    }

    return [expr {$length / 2}]
}

# msgcat::mcflmset --
#
#	Set the translation for multiple strings in the mc file locale.
#
# Arguments:
#	pairs		One or more src/dest pairs (must be even length)
#
# Results:
#	Returns the number of pairs processed

proc msgcat::mcflmset {pairs} {
    variable FileLocale
    variable Msgs

    if {![info exists FileLocale]} {
	return -code error \
	    "must only be used inside a message catalog loaded with ::msgcat::mcload"
    }
    set length [llength $pairs]
    if {$length % 2} {
	return -code error "bad translation list:\
		should be \"[lindex [info level 0] 0] locale {src dest ...}\""
    }

    set ns [uplevel 1 [list ::namespace current]]
    foreach {src dest} $pairs {
	dict set Msgs $FileLocale $ns $src $dest
    }
    return [expr {$length / 2}]
}

# msgcat::mcunknown --
#
#	This routine is called by msgcat::mc if a translation cannot
#	be found for a string.  This routine is intended to be replaced
#	by an application specific routine for error reporting

if {![package vsatisfies [package provide Tcl] 8.5]} {return}
package ifneeded msgcat 1.5.0 [list source [file join $dir msgcat.tcl]]

# message catalogs for locales foo, foo_BAR, and foo_BAR_baz.

package require Tcl 8.2
if {[catch {package require tcltest 2}]} {
    puts stderr "Skipping tests in [info script].  tcltest 2 required."
    return
}
if {[catch {package require msgcat 1.5.0}]} {
    puts stderr "Skipping tests in [info script].  No msgcat 1.5.0 found to test."
    return
}

namespace eval ::msgcat::test {
    namespace import ::msgcat::*
    namespace import ::tcltest::test
    namespace import ::tcltest::cleanupTests

    foreach setVars [PowerSet $envVars] { 
	set result [string tolower [lindex $setVars 0]]
	if {[string length $result] == 0} {
	    if {[info exists ::tcl::mac::locale]} {
		set result [string tolower \
			[msgcat::ConvertLocale $::tcl::mac::locale]]
	    } else {
		if {([info sharedlibextension] == ".dll")
			&& ![catch {package require registry}]} {
		    # Windows and Cygwin have other ways to determine the
		    # locale when the environment variables are missing
		    # and the registry package is present
		    continue
		}
		set result c
	    }
	}
	test msgcat-0.$count [list \
	    locale initialization from environment variables $setVars \
	] -setup {
	    variable var

	variable locale [mclocale]
	mclocale foo
    } -cleanup {
	mclocale $locale
    } -body {
	mc "this is a %s" "good test"
    } -result "this is a good test"

    # Tests msgcat-8.*: [mcflset]

    set msgdir1 [makeDirectory msgdir1]
    makeFile {::msgcat::mcflset k1 v1} l1.msg $msgdir1

	test msgcat-8.1 {mcflset} -setup {
	    variable locale [mclocale]
	    mclocale l1
	    mcload $msgdir1
	} -cleanup {
	    mclocale $locale
	} -body {
	    mc k1
	} -result v1

    removeFile l1.msg $msgdir1
    removeDirectory msgdir1

    set msgdir2 [makeDirectory msgdir2]
    set msgdir3 [makeDirectory msgdir3]
    makeFile "::msgcat::mcflset k2 v2 ; ::msgcat::mcload [list $msgdir3]"\
	    l2.msg $msgdir2
    makeFile {::msgcat::mcflset k3 v3} l2.msg $msgdir3

	# chained mcload
	test msgcat-8.2 {mcflset} -setup {
	    variable locale [mclocale]
	    mclocale l2
	    mcload $msgdir2
	} -cleanup {
	    mclocale $locale
	} -body {
	    return [mc k2][mc k3]
	} -result v2v3

    removeFile l2.msg $msgdir2
    removeDirectory msgdir2
    removeFile l3.msg $msgdir3
    removeDirectory msgdir3

    cleanupTests
}
namespace delete ::msgcat::test
return

	@echo "Installing package http 2.7.9 as a Tcl Module";
	@$(INSTALL_DATA) $(TOP_DIR)/library/http/http.tcl "$(SCRIPT_INSTALL_DIR)"/../tcl8/8.4/http-2.7.9.tm;
	@echo "Installing package opt0.4 files to $(SCRIPT_INSTALL_DIR)/opt0.4/";
	@for i in $(TOP_DIR)/library/opt/*.tcl ; \
	    do \
	    $(INSTALL_DATA) $$i "$(SCRIPT_INSTALL_DIR)"/opt0.4; \
	    done;
	@echo "Installing package msgcat 1.5.0 as a Tcl Module";
	@$(INSTALL_DATA) $(TOP_DIR)/library/msgcat/msgcat.tcl "$(SCRIPT_INSTALL_DIR)"/../tcl8/8.5/msgcat-1.5.0.tm;
	@echo "Installing package tcltest 2.3.4 as a Tcl Module";
	@$(INSTALL_DATA) $(TOP_DIR)/library/tcltest/tcltest.tcl "$(SCRIPT_INSTALL_DIR)"/../tcl8/8.5/tcltest-2.3.4.tm;

	@echo "Installing package platform 1.0.10 as a Tcl Module";
	@$(INSTALL_DATA) $(TOP_DIR)/library/platform/platform.tcl "$(SCRIPT_INSTALL_DIR)"/../tcl8/8.4/platform-1.0.10.tm;
	@echo "Installing package platform::shell 1.1.4 as a Tcl Module";
	@$(INSTALL_DATA) $(TOP_DIR)/library/platform/shell.tcl "$(SCRIPT_INSTALL_DIR)"/../tcl8/8.4/platform/shell-1.1.4.tm;

	@echo "Installing package http 2.7.9 as a Tcl Module";
	@$(COPY) $(ROOT_DIR)/library/http/http.tcl $(SCRIPT_INSTALL_DIR)/../tcl8/8.4/http-2.7.9.tm;
	@echo "Installing library opt0.4 directory";
	@for j in $(ROOT_DIR)/library/opt/*.tcl; \
	    do \
	    $(COPY) "$$j" "$(SCRIPT_INSTALL_DIR)/opt0.4"; \
	    done;
	@echo "Installing package msgcat 1.5.0 as a Tcl Module";
	@$(COPY) $(ROOT_DIR)/library/msgcat/msgcat.tcl $(SCRIPT_INSTALL_DIR)/../tcl8/8.5/msgcat-1.5.0.tm;
	@echo "Installing package tcltest 2.3.4 as a Tcl Module";
	@$(COPY) $(ROOT_DIR)/library/tcltest/tcltest.tcl $(SCRIPT_INSTALL_DIR)/../tcl8/8.5/tcltest-2.3.4.tm;
	@echo "Installing package platform 1.0.10 as a Tcl Module";
	@$(COPY) $(ROOT_DIR)/library/platform/platform.tcl $(SCRIPT_INSTALL_DIR)/../tcl8/8.4/platform-1.0.10.tm;
	@echo "Installing package platform::shell 1.1.4 as a Tcl Module";
	@$(COPY) $(ROOT_DIR)/library/platform/shell.tcl $(SCRIPT_INSTALL_DIR)/../tcl8/8.4/platform/shell-1.1.4.tm;
	@echo "Installing encodings";