View Ticket
Not logged in
Ticket UUID: 9c6ff35e394cfed95235b8a7871cb340135bf940
Title: tclws (2.4?) changes to support REST/JSON, optional arguments, and more
Status: Closed Type: Feature_Request
Severity: Critical Priority: Immediate
Subsystem: Server_Side Resolution: Fixed
Assigned to: gerald
Last Modified: 2017-09-07 13:06:21
Version Found In: 2.3.8
Description & Comments:
As a part of FlightAware's use of tclws (Web Services for Tcl), we have developed some significant improvements to tclws that allow it to support returning REST/JSON responses from any function that had previously been SOAP-only.

Branch fwr-jsonrest-changes

All of these changes are currently in this branch for a proposed version 2.4 of tclws: http://core.tcl.tk/tclws/timeline?r=fwr-jsonrest-changes

Optional WSDL structure member using '?'

Included is support for optional structure members (minOccurs=0 in the WSDL) if you define the datatype with a suffixed "?" character. (This is in the spirit of the existing tclws datatype syntax of using the "()" suffix to indicate arrays, and also in the style of Swift optional arguments.)

Verify return of callback procs

Additionally, if you specify -enforceRequired Y in the service definition you will receive an error if any of your procs accidentally forget to return a structure member that had not been designated optional. The default is -enforceRequired N, which will preserve the prior behavior of simply returning the result anyways with the missing members absent.

Basic input validation

Basic input argument validation can be requested by specifying -verifyUserArgs Y in the service definition.

Customize error return

We've also added a -errorCallback option to the service definition to allow your code to customize any exceptions before they are actually sent back to the caller. (We use this functionality to internally log caller-induced errors and to customize the HTTP response headers and status codes in certain cases.)

REST/JSON

Service documentation can also be emitted into a JSON format, rather than only HTML or WSDL. We use this to provide a more complex client-side presentation of the documentation than would be possible with only custom CSS against the HTML: https://flightaware.com/commercial/flightxml/explorer/

flavor variable to activate REST

Internally, we've changed many of the procs in ::WS::Server to define a new "flavor" variable throughout that specifies "soap" or "rest" mode, and made ::WS::Server::callOperation accept that flavor as an optional argument. If the flavor is not specified, then soap is still assumed by default in order to preserve compatibility with existing tclws users.

Requires yajl

In order to serialize the JSON responses, we added a dependency on our yajl-tcl package, but that is only required when you initiate a request in the "rest" flavor: https://github.com/flightaware/yajl-tcl

No JSON serialized data parsing

Our use case has only required us to accept FORM arguments and return JSON responses for everything, so we haven't implemented logic to parse any input arguments that are passed in as JSON serialized data, but this might be an area of future exploration for someone.

Rivet example

Here's a bit of code showing how we initially start up this mode in Apache Rivet, which is actually pretty similar to how you'd use tclws in SOAP mode from Apache Rivet:

        # Capture the info from the request into an array.
        load_headers hdrArray
        set sock [pid];         # an arbitrary value
        array unset ::Httpd$sock

        # Prepare the CGI style arguments into a list
        load_response formArray
        set opname $formArray(call)
        unset formArray(call)
        set queryarg [list $opname [array get formArray]]

        # Invoke the the method
        array set ::Httpd$sock [list query $queryarg ipaddr [env REMOTE_ADDR] headerlist [array get hdrArray]]

        # Invoke the method in REST mode.
        set result [catch {::WS::Server::callOperation $svcname $sock -rest} error]
        array unset ::Httpd$sock
        if {$result} {
                headers numeric 500
                puts "Operation failed: $error"
                abort_page
        }

oehhar added on 2017-08-30 09:03:59 UTC:
I am not sure if we have two meanings for the word "REST".

  • return flavor returning a JSON document instead of XML
  • Type field in "::WS::Client::CreateService" which apparently uses REST http methods but no JSON (as far as I understand)

So, I am not so much in favor to call the JSON return type "REST". I would prefer "JSON".


oehhar added on 2017-09-07 13:06:21 UTC:
I am adding here a communication between Gerlad, Jeff and Rolf (tdom) dated 2017-07-24, targeting to not require jalil library:

From Gerald

It turns out Rolf is adding JSON "support" (for a suitable definition of the word "support") to TDOM -- could one or both of ya'll get with him to express what is needed to allow TclWS to use this feature and not have to deal with JSON using the other library that Jeff's changes are using.

BTW, think of how this might be able to give us full JSON support for TclWS.

From Rolf

Gerald pushed me to point you at

http://core.tcl.tk/tdom/artifact/1061d0365930fe9b

Goal, at least so far, is to have a JSON to (somewhat friendly) XML mapping, with full turn-around JSON -> XML -> JSON. This will not provide / support a (any random) XML -> JSON -> XML path. Though, with the "right" from the scratch build DOM tree you will be able to generate any desired JSON.

At least, that's the plan.

From Jeff

I would be interested in seeing performance benchmarks for parsing of large JSON inputs. One of the key benefits of our yajl-tcl library is the efficiency of parsing and serializing, because it uses the high-performance yajl C library for the underlying operations. I see that your tdom implementation also appears to implement its own JSON parser in C, so the performance might be quite comparable.

N.B.

The required feature got available with tdom 0.9.0 release.