Tcl Library Source Code

Hi Andreas,

As briefly discussed yesterday on IRC, I am interested in getting FFP tracking/reporting into the PARAM machinery. Right now, an FFP heuristic is applied in error handling, but it is confined within a given context (a choice). An overall (global) FFP is not maintained for a parse.

For the NX backend (::pt::rde), I got an FFP extension (see script attached). I wonder, however, how this would be best become integrated into the base machinery, as all backends would benefit from this (practical error tracking in PEGs is much more convenient with an FFP hint). 

The main idea of the extension is to intercept on "choice exits", intermittent exits (inter*) and actual exits (exit0/1), and update an FFP store (updateFFP). The update logic is basically the same operation as within choices: i_error_pop_merge (but without any popping). It works as expected (see the Tiny language example in the script).

Qs:

- How can this best be supported in the base machinery?
- In my extension, the downside is that the intercepted SIs are not specific to  choices, but are also compiled into other parsing methods (sequences). So, the FFP update is triggered more often than necessary.
- What are the consequences for error reporting: Should [complete] be made FFP aware, somehow?

Let me know how I can help. That said, the extension works well for my needs. But I think PT users would benefit from this otherwise commonly available facility in its PARAM.

For the records, the chat protocol from yesterday:

m 2017-10-05T22:31:52Z ijchain {<mr_calvin> aku: around?}
m 2017-10-05T22:32:28Z aku pong
m 2017-10-05T22:32:48Z ijchain {<mr_calvin> regarding ffp for pt ...}
m 2017-10-05T22:33:19Z ijchain {<mr_calvin> I got a minimal version (truly minimal) based on the nx backend, to show to you}
m 2017-10-05T22:34:33Z aku {Oh, tcllib ticket ...}
m 2017-10-05T22:34:46Z ijchain {<mr_calvin> not yet a ticket :)}
m 2017-10-05T22:35:33Z aku {I believe I saw a ticket about `fail farthest p?` for pt::peg things ...}
m 2017-10-05T22:35:49Z ijchain {<mr_calvin> was an email :)}
m 2017-10-05T22:36:16Z aku {Ah. Ok, that is possible. I usually see ticket through mail notification, so might be scrambled there.}
m 2017-10-05T22:36:33Z aku {somethng to me ... best shown in email as well, I believe ...}
m 2017-10-05T22:39:55Z aku {I believe I will have to see how you did the instrumentation ....}
m 2017-10-05T22:40:58Z ijchain {<mr_calvin> (obviously one can extend the basic si’s to just do what I do in a few line in the overloading methods, but you wrote in your email that you prefer sth. orthogonal)}
m 2017-10-05T22:42:53Z aku {did I ... Suspect I have to reread ... I thought I said ... orthogonal first prefered, then integration ... If we need intregration we certainly can go there too ...}
m 2017-10-05T22:43:12Z aku {well, I will see what is in your mail.}
m 2017-10-05T22:44:13Z ijchain {<mr_calvin> downside of handling at this level of si’s that the ffp checks are done more often than necessary … triggering the ffp check “on error” is certainly more proportionate.}
m 2017-10-05T22:50:12Z aku {mr_calvin - Is that even possible (doing the check only on error ?)}
m 2017-10-05T22:54:12Z ijchain {<mr_calvin> well, “on error” in the sense of the i_error_pop_merge limbo …}
m 2017-10-05T22:55:12Z ijchain {<mr_calvin> actually, right now, I try to capture what the inlined i_error_pop_merge logic does at the exit of a choice ...}
m 2017-10-05T22:56:41Z ijchain {<mr_calvin> the thing is, that the si’s to do so are also fired when exiting sequences etc. it is not obvious to me how to limit it to “choice exits” …}
m 2017-10-05T22:56:58Z ijchain {<mr_calvin> so I should have written “on choice exit” :)}
m 2017-10-05T22:58:19Z aku {mr-calvin - ok, makes sense ... might new special si specific to choices then ...}
m 2017-10-05T22:58:28Z aku {s/new/need new/}
m 2017-10-05T22:58:37Z aku {put this all into the mail too, please}

Seeing ticket [351b8b2f55] now for the transform issue, closing this one as complete.

Ok, and merged. Commit [623801c583].

Running the other branch (pt-container-ssoberni) through this ticket as well, keeping it open.

Just pushed a commit on the branch which fixes the tests.

See [d58dafb1140c96d6].

It uses the "!." construction to force the parser to reject anything which does not consume the entire input ("!" = negative look-ahead, "." = any character, i.e. at the given point are "no more characters allowed").

After that I fixed the ok example. It was a copy of the fail example, so I just had to add the missing semicolon.

In the error results the report location was not quite right, got fixed.

Due to my grammar changes I had to update the ok result as well.

If you like that I will merge into trunk.

As hinted at in IRC, it turned out that the FFP was erroneously dropped due to using the equivalent of i_error_clear_push rather than of i_error_push in three out of four *_branch (choice) super-instructions:

si:voidvalue_branch
si:valuevoid_branch
si:valuevalue_branch

This was fixed for the TclOO and Snit RDE (the C one was not affected, naturally). NX benefits from the TclOO fix directly, as it derives from the TclOO engine.

Branch ssoberni-ffp-3ed39a451f [*] contains a fix plus test. The test is WIP, because I don't exactly know how to best instrument the test runner to cover for this failure case? Here, the combined "failure" condition would be (as signalled by parset):

- matched prefix ne input /xor/ suffix ne ""
- the content of myerror with the FFP

Pls. advise.

[*] https://core.tcl.tk/tcllib/timeline?r=ssoberni-ffp-3ed39a451f