Token on line after end of docstring causes parser confusion and 'useless error'

[hopscc]

Also found by Chasm.
see  Useless Error

Token(s) after a closing docstring token causes end of docstring to be unrecognised so text is then absorbed to start of next docstring.
first two tokens of that docstring look like a reversed var declaration which causes the 'local variable' message.

Suggest adjusting end of docstring token to allow (but ignore and warn on) non wspace to eoln after closing (CobraTokeniser)

(I downrated this slightly to medium cos its a pretty obscure situation (accidental) but the resulting error message and source fingering is utterly unhelpful.)

[Charles]

If I'm reading your suggestion right, you mean that his example would be allowed with a warning, but instead, it should be an error.

[hopscc]

No - ignore or ignore with warning.
Either (silently) allow his example ignoring the trailing tokens after the closing """ or allow the example, ignore the tokens but emit a warning message....

I dont think this is an error situation more an 'indication of interpretation' one.
(multiline docstrings should have leading/trailing """ on own lines - we will interpret closing """ with trailing text that way (and perhaps warn)
its not an error but either way cobra is ignoring the trailing text...).

( sorry I originally wrote this thinking we should just ignore trailing stuff cos its wrong "multiline docstrings ... on own lines")
but then thought its friendlier to actually notify that this is the interpretation we're taking).

[Charles]

That doesn't change my perspective. It's a syntax error.

[hopscc]

Yes it is a syntax error - we have established that.

The question is, given that we can detect and recover from it
(ignore the trailing tokens), do we

a) silently ignore the (incorrect) trailing tokens - writer *may* silently lose some code, probably more likely just have some ignored trailing crud after their docString
b) noisily ignore the (incorrect) trailing tokens - emit a warning and continue.
c) throw an error and stop parsing/tokenising even though we could continue and pick up any further problems...

probably comes down to b) or c) since theres all sorts of good reasons not to ignore some code silently...

[Charles]

d) record an error and continue

Fixed locally; running tests.

[Charles]

changeset:3022