Tcl Source Code

In porting an existing package based on C to one
that uses Tcl as the glue, I've had the desire to
have the Tk_ParseArgv functionality without Tk.
This lead me to find the following old conversation
regarding this:

http://www.google.com/url?sa=D&q=http://groups.google.ca/group/comp.lang.tcl/browse_thread/thread/c4fea8f0346cf8ae/036961bf476a3b99%3Flnk%3Dst%26q%3DTcl_ParseArgv%26rnum%3D1%23036961bf476a3b99

The arguments against Tcl_ParseArgv is that if you are
using tcl, then just use it as the main loop, and call
out to special function C codes as necessary. By doing
so, you can use one of the many Tcl Command line
parsing tools listed at http://wiki.tcl.tk/1730.

Now while the arguments against this functionality are
valid within the context of the process' main loop, it
breaks down when you want to parse options in
stand-alone commands. At least this is how I see
it.

To be more precise, I have a C library to which I want
to provide a Tcl interface.
So for each of the functions in the library, I need to
write a wrapper in C, to provide that command to Tcl.
If I want these commands to have libpopt or Tk-like
command options I either have to
  1) Parse the command arguments myself (on the C side); or
  2) Write additional wrappers to the commands in Tcl,
to pre-parse the options.
So in the case of (2), I'm making work for myself, when
all I really want to be able to do is call
Tcl_ParseArgv and be done with it in the original C
command wrapper.

NEWays, to get things working quickly, I wrote a
Tcl_ParseArgv routine, based (of course)
heavily on the tkArgv.c source (which implements
Tk_ParseArgv), but with all the Tk specific bits taken
out.  So now my wrappers can be ported very quickly
from their current libpopt handled options, to the
Tcl_ParseArgv.

Summary:
 1) Provide a Tcl_ParseArgv routine that can parse
    a standard argc,argv[] pair.
 2) Useful when functions other than main() wish
    to parse arguments in argc, argv[] format.
 3) Useful when one wishes to have libpopt style
    command line arguments to functions.
   3a) Build argc,argv from objc,objv.
   3b) Parse using Tcl_ParseArgv.
   3c) Parse remaining arguments in familiar C fashion.
 4) Makes re-writing of C-based wrappers written using
    libpopt trivial.

Code (which can be used as a shared library),
is attached.

IP - Comment Removed: 130.88.1.31

data_type - 210894

This was TIP#265, implemented in 8.6

JVbEWr dkg93jfbkSdLk496c

File Deleted - 170441:

File Deleted - 184870:

File Added - 184871: tclArgv-tcl8.4.13_20060713.patch

Logged In: YES 
user_id=1472195

Hi,
I've created a patch against the Tcl 8.4.13 sources as
you requested. I've made most of the changes that were
suggested. One thing outstanding is the addition of
support for all of the TCL_LINK* types. As per Don's
suggestion, the default help table has been removed,
remaining arguments are now gathered if requested. and the
flags argument has been dropped.
Comments are most welcome. I'll do my best to get this into
a form that is acceptable.
Sam

File Added - 184870: tclArgv-tcl8.4.13_20060713.patch

Logged In: YES 
user_id=1472195

Hi,
I've created a patch against the Tcl 8.4.13 sources as
you requested. I've made most of the changes that were
suggested. One thing outstanding is the addition of
support for all of the TCL_LINK* types. As per Don's
suggestion, the default help table has been removed,
remaining arguments are now gathered if requested. and the
flags argument has been dropped.
Comments are most welcome. I'll do my best to get this into
a form that is acceptable.
Sam

Logged In: YES 
user_id=1472195

Hi Stephen,
Thanks for your reply. It looks like a lot of thought
has been put into tclobjv.c, and perhaps it would make
a better codebase to adopt into Tcl proper, tho I don't
yet really have the Tcl experience to judge. :)

If you would indulge me a little further, I'd like to raise
a few more questions with you. In my use of tclArgv.c, we've
implemented many commands, I guess effectively creating
somewhat of a domain specific language. A typical command
actually performs many functions and so we've found that our
commands often have tens of options. Remembering them all
is hard without an automatically generated help. Basically I
feel that the existing GetIndexFromObj suffices for typical
C-implemented Tcl commands, and in this context, an
automaticly generated help is extraneous. However, I figure
that if you need the extra complexity of tclArgv for your
command parsing, then you'd probably want the automatically
generated help. Being able to add descriptions to each
option is then a real benefit. Would you consider adding the
ability to attach descriptions to each of your options and
arguments? (I really like the way you handle the arguments
as well, as currenly, tclArgv does not generate automatic
help for the arguments, but strapping this onto tclobjv
could be a nice way to go.)

Requiring a specific order for arguments (that is, coming
before all options) is probably a good idea for consistency.
Sure tclArgv doesn't care, but any user could get used to it.

In one sense I like that your unknown arguments trigger an
error. On the other hand, this prevents you from calling a
nested command with the same objv[]. Doing this kind of
thing is probably not very common outside of X application
programming, so I'm sure we need not worry about it.

I like that you already have a complete set of tests
available. It is also good that you use Tcl_GetIndexFromObj.

Aestetically (which means a trivial change only is required)
I would like to see a version that removes all the "Ns"
prepending so that it looks more core-Tcl-ish. (Yeah, this
point is obvious. :) )

W.r.t. to Don's suggestions for tclArgv, I've made the
following changes:
(i) Removed TCL_DONT_SKIP_FIRST_ARG.
(ii) TCL_ARGV_NO_ABBREV -> TCL_EXACT
(iii) Added a &remObjv argument to collect unprocessed
arguments, and thus also eliminating the
TCL_ARGV_NO_LEFTOVERS flag.
(iv) I disagree with ditching the -help option, for the
reason I cited above regarding the developer/user already
choosing the greater complexity of tclArgv over a
Tcl_GetIndexFromObjv-only solution.

I'm still continuing work on it. I haven't yet re-cast
Tcl_ParseArgv as a thin wrapper around Tcl_ParseObjArgv,
nor added the missing data types. I'll do my best to submit
a patch with all the changes soon.

Meanwhile, if Stephen could provide me with a little package
that allowed me to use his version as a little shared
library, with the modification to provide dynamic help, I'll
merge it in with my application and give it a test drive to
see what the users think.

I look forward to hearing any suggestions from anyone
regarding the suitability (or lack-thereof) of the two
options (TclArgv and tclobjv).

Cheers,
Sam.

Logged In: YES 
user_id=87254

Hi Sam.  I wrote the anonymous comment (sure I was logged in
-- silly source forge) and tclobjv.c in naviserver.

Yes, tclobjv.c parses both options and arguments, handling
type conversion and defaults for both, and an optional
'args' to gather remaining arguments in. It does this with
the Ns_ObjvArgs callback.  There's lots of examples of it
being used here:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclsched.c?revision=1.5&view=markup

You can parse the remaining arguments however you like.


> (i) Does the order of options matter?

No, which is the expected behaviour, I think.


> (ii) Can options be interspersed with arguments?

No. All options must come before arguments, which again I
think is the expected behaviour.


> (iii) What happens in the case of unknown arguments?

Not sure what you mean here.  If the args supplied don't
match the signature, an error message is generated. If you
expect an argument of an unusual type, you can use
Ns_ObjvObj and the Tcl object will be returned, which you
can interpret however you like.


> (iv) Is there an automatically generated help?

Error messages are automatically generated which descibe the
command's signature. Perhaps the signature should be made
available, so that commands could implement a -help switch
(although this feels wrong to me for commands, fine for
command lines).


> (v) Do all arguments have to be known in advance?

The number of remaining, unparsed arguments can be captured
and dealt with however you like (see above).



tclobjv.c is the fastest implementation I could come up with
(standalone, not including the Tcl byte code compiler),
which is important for Tcl commands, though probably not for
application command lines.

Some things which tclobjv.c does to make things fast are: no
memory is allocated for parsing; uses GetIndexFromObj to
shimmer switches to speed parsing in the future; splits
option and arg parsing into two phases so that un-passed
options can be skipped efficiently; direct jump to type
conversion callbacks rather than giant switch statement. 
The last is obviously more felxible, too.

The core Ns_ParseObjv() is only ~45 lines. The type
conversion callbacks are often trivial wrappers.


There's also a complete set of tests available:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/tests/ns_parseargs.test?revision=1.3&view=markup

Logged In: YES 
user_id=80530


For me, it would be easier to review
the code if it were a patch against
the Tcl sources.  Stick the new routines
in tclIndexObj.c.  Follow the engineering
manual, please.

The set of types indicated by
TCL_ARGV_* is incomplete.  Look
at the full list of TCL_LINK_*
types now supported by Tcl_LinkVar.
In fact, can we just have these
routines just re-use the TCL_LINK_*
values?

The Tcl_ParseArgv() and
Tcl_ParseArgvObj() routines are
separate implementations.  Seems
to me it would be better to
implement one, and make the other
a thin wrapper around it.  I'd
have the Tcl_Obj one be real, and
the string-based interface as a
wrapper, if we even bother to have
it at all.

The TCL_DONT_SKIP_FIRST_ARG flag
seems pointless.  Just expect the
caller to pass in the correct value
for objv.

It's not clear to me that having
a "-help" option always recognized
is more useful than it is annoying.
Who are the expected callers of
these routines, and are they expected
to want a "-help" more often than not?
If we drop that, we drop the whole
defaultTable, as well as the need for
the TCL_ARGV_NO_DEFAULTS flag.

TCL_ARGV_NO_ABBREV has the same meaning
as the existing TCL_EXACT, right?  Why
not re-use that?

Rather than require objv to be over-writable,
I think I'd be happier with a routine
that took a const objv and an optional 
&remainingObjv argument, for sake of those
callers that want to know the unparsed
options.  With that revision, it would
be possible for the caller to manage
the reaction to whether or not all objv
values are consumed, so TCL_ARGV_NO_LEFTOVERS
would no longer be needed either.

I think that does away with the need
for the flags argument entirely.

I guess that's enough for now.

Logged In: YES 
user_id=1472195

Yep. Naviserver's tclobjv.c seems to provide similar
functionality, without the need to make a local copy of the
objects. It gets around this by parsing both options and
arguments in the call, whereas the tclArgv version (based on
TkArgv.c) parses only the options, and repacks the array
leaving only arguments and unknown options. This ability to
leave unknown arguments and options on the objv array is
useful when one wants to strip different options with
different functions (though I admit, I haven't needed to do
so myself as of yet.)

From a usability perspective I'm sure both approaches are
valid. I have a few questions that could help bring me up to
speed on the naviserver version:
(i) Does the order of options matter?
(ii) Can options be interspersed with arguments?
(iii) What happens in the case of unknown arguments?
(iv) Is there an automatically generated help?
(v) Do all arguments have to be known in advance?
Equivalently, can the number of arguments be dynamic,
depending on, say, the value of options? (tclArgv allows
this as the arguments are just repackaged for subsequent
parsing by the caller).

My bias (at least in terms of for inclusion
into Tcl proper) is that the function be as stand-alone and
as simple as possible. In this sense, the tclArgv.c file
below might be seen as a more closed-form solution than the
tclobjv.c version with its additional typedefs and
subfunctions. Moreover, tclArgv.[hc] are the only two files
needed to add this functionality to any project (which we
use  quite cleanly now compiled up as a shared library).
Perhaps the naviserver tclobjv could be easily simplified as
well. I welcome comments on these points.

Should tclArgv.c be deemed sufficiently useful for
consideration for inclusion into Tcl proper, I'd be happy to
recode a GetIndexFromObject version of tclArgv.c if this is
a sticking point.

Logged In: NO 

Please take a look at this version, which does use
GetIndexFromObject, parses args as well options, and uses
callbacks for type checking:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclobjv.c?view=markup


There's lots of examples of it being used, e.g. here:

http://naviserver.cvs.sourceforge.net/naviserver/naviserver/nsd/tclrequest.c?revision=1.14&view=markup

File Added - 181577: tclArgv-20060613.tar.gz

Logged In: YES 
user_id=1472195

The version I wrote using Tcl_GetIndexFromObj was causing a
crash. I've reverted to my original version in the latest
version (20060613), and it is this version that I am
currently using in production. I've included an example code
that compiles a Tcl extension that you can load using
"load", and it provides the "parseme" example command. In
addition to the coded options, you also have automatic help
via "-help".

I will be happy to assist in any way you see fit.

Logged In: YES 
user_id=80530


I want to make a real patch
file for this.  Which of the
attached files are the right
one(s) to use?

File Added - 173278: tclArgv.c

Logged In: YES 
user_id=1472195

W.r.t. the use of Tcl_GetIndexFromObj(), I did consider
it, and originally decided that it was not the way to go.
If I wished to implement a single command, I would surely
have used it, however, the benifit of the tclArgv approach,
is that the writer of the C implemented command can quite
cleanly package everything up into a single array of
Tcl_ArgInfo structures, and specify
type checking and conversion all in one neat place.

So,I originally opted for the simplicity of the current
code. I have however now rewritten it to use
Tcl_GetIndexFromObjStruct, to remove the parsing bit,
as per your suggestion. I hope this is less bizarre. :)

Logged In: YES 
user_id=80530


Took a quick glance at the
submitted code and I see no
use of Tcl_GetIndexFromObj().
Seems very bizarre to me,
as I'd expect any command line
parsing to be a thin wrapper
around it.

Logged In: YES 
user_id=79902

Requires a TIP. Email me for details on the process.

File Added - 170511: tclArgv.tar.gz

Logged In: YES 
user_id=1472195

I've added a routine "Tcl_ParseArgsObj" that
provides libpopt or Tk_ParseArgv type parsing,
but for Tcl_Obj *objv[] array passed to
C implemented Tcl commands.

File Deleted - 170362:

File Deleted - 170364:

File Added - 170441: Tcl_ParseArgv.tar.gz

File Added - 170364: Tcl_ParseArgv.tar.gz

File Added - 170362: Tcl_ParseArgv.tar.gz

File Added - 170359: Tcl_ParseArgv.tar.gz

File Added - 170358: Tcl_ParseArgv.tar.gz

File Added - 170355: Tcl_ParseArgv.tar.gz

File Deleted - 170354:

File Added - 170354: tclArgv.h