More support for Java BackEnd

[hopscc]

Placeholder ticket for patches to cobra for support of Java back end.

-back-end:jvm

Initially support compilation using javac in path and execution of resultant class file(s) (java in path).
Flesh out JavaGenerator so basic tests compile and run correctly.

I'll be uploading a rolling patch for this, later patches will replace/supercede/include earlier patches.
Probably code changes only, no additional tests.

[Charles]

You may be interested in  http://www.ikvm.net/

[hopscc]

Initial patch for java backend. (Cross compilation)

Code to support multiple back ends, push platform dependent actions to a backend.
Find/run java and javac ( in path).
Map cobra filenames to java filenames ( on basis of first class in cobra files).
Stub RTL ( in java)
Support tool for generating java library ( jar files) class descriptions ( pkgSig file)
java back end code, read/parse pkgsig file and generate JavaClassType? representations.
Scanning/loading of representations for checking against as referenced..
Some scripts for creation and install and doc for new java files...

[Charles]

Patches applied. Some minor renames:

Source/Cobra.Lang/java --> Java  # capped
00-README --> read-me.text
TODO --> to-do.text

I have only retested the .NET compiler. If I missed something in the JVM patch or messed it up, let me know.

[hopscc]

Second patch - contains

• cleanup original patch code esp branching and use of scanNativeType for use of backend object
• suppress some debug output
• push more platform stuff to back end
• (More) better support for Java Generics.
• Fixes to PkgSig (signature generation)

• For tests extend '.require. mono| dotnet' to allow also 'clr' and 'jvm' so can write tests specific to a backend (using java libs rather than .Net ones for example) - provide a runtimePlatform string in RTL

• Started adding some backend specific tests

• support for properties, recognised but code gen incomplete (setters)
• disable last patch recognition code for indexors - pattern approach is not viable

[Charles]

Applied.

[hopscc]

Third patch - support for setting a property, Delegates baseclass, fix for assert
additional Test.

[Charles]

Thanks. Applied in changset:2587

[hopscc]

4th patch.
Create, Extend and add min tests for Delegates base class (reqd for setProp)
Start of Cobra Exceptions conversion
codegen for missing literals, various Math expressions (Float and Double)
tests and fixups for assert, bool,Int and Decimal and Float tests ( Tests working thru to 060-string (needs Indexors))
Start of handling for Cobra Decimal (-> Double)
Minor property tweaks to CobraCore and CobraImp lib code.
Add flags to support marking main method and mainMethod (static) wrapper so codegen can do right thing.

Fix for lurking (off-by-lines-processed) bug in Testify runner surfaced by more use of
#.require test directive

[hopscc]

As mentioned in forum discussion
Patch to push mainWrapper code generation to explicit code emission in {Sharp,Java}Generator.cobra and simplification in the front end general code.

Not specific to Java but will make future back end main handling easier.

[Charles]

Ug, I've lost track. Which of these patches is outstanding? I presume it's *4.patch and main-wrapper-be.patch since I don't have comments after that.

Is that correct?

Maybe in the future we should just kick out one ticket per patch.

[hopscc]

Its my understanding that the patches presented here are all applied to the source base
i.e you are up to date with this ticket

the *-4 patch was applied with changeset:2592
(see the forum thread )
The main wrapper rework was applied with changeset:2607 by the look of it.

You'd help yourself with staying on top of this if you remembered/continued to note in the ticket the changeset a patch was applied after the patch was done

(see exchanges on this ticket 5+ 6 postings ago:

• patch posted (java-jvm-part3.patch), patch applied and changeset (2587) noted following )

• for the *-4 patch I think this got confuselated cos there was a notification posted to the discussion forum topic ( http://cobra-language.com/forums/viewtopic.php?f=4&t=722&start=30#p4323) instead.

With respect, providing a new ticket per patch for an ongoing/rolling feature enhancement where the patches stack on each other is just stupid...
( e.g. imagine what happens if a patch/ticket gets missed application)

[Charles]

I was already aware that updating the ticket with changesets is useful as demonstrated by my past behavior which you noted. As with any useful practice, one might forget in a particular instance.

I have no idea why you think the one patch per ticket suggestion is stupid. The only thing you mentioned in support of that doesn't seem plausible to me. Imagine that when I look at the "tickets recently updated sans closed" report, I'll see any relevant JVM patches including ticket number/order. That's the report I always use.

[hopscc]

Note the proviso - Ongoing feature/enhancement development where patches stack on each other - at a minimum separate patches for each piece don't indicate previous dependencies.

For the plausibility of the possibility of missed application of (simpler single issue non dependent) tickets+patches see this Custom Query.
I'd imagine that all of these appeared/appear (high up at some point) in the report you mention.

[hopscc]

5th patch.

CommandLine access for generated Java sources.
Readable (indented) TechStrings triggered by COBRA_INDENT_TECHSTRING environment variable.
Add .platform. compiler directive to allow wiring cobra source to a backend

    e.g @platform clr
        @platform jvm

Virtualise support for Types that java doesnt have ( e.g. uint) and remap at lowest level so cobra treats them as distinct.
More dancing around for handling Java Generics.
Rework properties for handling Indexers, Wire in simple Indexer support.

Exceptions and SourceSite Implementations in Java.
Java checkNonNil and toTechString Implementation.

Modified Tests wired to clr and jvm specifically.

[Charles]

Thanks. Applied in changeset:2631

See also:  http://cobra-language.com/forums/viewtopic.php?f=4&t=722&p=4508#p4508

[hopscc]

Been a while since the last patch so thought I should resync.

Unfortunately this patch is only incremental minor changes - mainly code cleanups/portability changes and tweaking. Biggest enhancement is to
get -testify working in conjunction with -backend setting and some consequent additions/changes to some tests (Unfortunately this is also indicating some test regressions ( java only) I wasnt aware of that I'll be addressing next).

Chuck, when you apply this patch could you also remove any of the *.sig files in Source/Cobra.Lang/Java that may still be under VCS control so that they are not - I think the only one (left) is CobraLang.jar.sig.
These files should all be autogenerated on install (eventually) or manually using one one of the mkJar scripts - either way they dont need to be under source control.

[Charles]

Patch 6 applied in changeset:2636

Nice catch on:

rtPlatform  = CobraCore.runtimePlatform
# fixed to:
rtPlatform = options['back-end']

Why all the changes from:

-# .require. jvm

to:

+#.require. jvm

and for clr too?

Did testify change? (Or not recognize those in the first place. I'm tired right now.)

Prior to the changes, testify gave me:

950 Tests in 00:19:58.4178090.
6 Failures.

And now gives:

929 Tests in 00:17:05.3573100.
2 Failures.

I haven't looked yet at where 31 tests went... (Still tired right now.)

[Charles]

CobraLang.jar.sig is now removed per your request.

I still see a BackEndJvm?/jarSigTestFile1.sig if you want that one deleted as well.

[hopscc]

Thx

For some reason I think I changed the rtplatform setting earlier and just noticed this round (with testify) it was wrong

.require changes
I thought at one stage testify wasnt recognising '# .require.' and needed '#.require.'
(no leading space) but turned out I'd been using '#.require' ( no trailing '.')
which I corrected without reverting the leading space ( since it makes no difference)

I handnt noticed the testify count change - I'll check it out
Dont believe there's anything that should change in testify that should do this ....

Leave the BackEndJvm/jarSigTestFile1.sig in VCS for now - its intended to be part of the test suite.

Forgot to mention - patch added a comment in Source/Types.cobra in isDescendantOf
querying the code in the conjunction above ( or rather than and) ??
Shouldnt the test be base is type OR wrappedType is type,

since if its a wrapped type the wrapped type base will be that of the wrapped envelope ??

[hopscc]

Grr. Irony

The rtPlatform line indicated above is still wrong - it was skipping all the
tests marked as requiring clr (on clr)

heres a patch to fix that

Index: Source/TestifyRunner.cobra
===================================================================
--- Source/TestifyRunner.cobra  (revision 2637)
+++ Source/TestifyRunner.cobra  (working copy)
@@ -471,7 +471,7 @@
                                continue

                        if firstLineInsensitive.startsWith('#.require.')
-                               rtPlatform = options['back-end']
+                               rtPlatform  = if(options['back-end']=='none', CobraCore.runtimePlatform, options['back-end'])
                                what = firstLineInsensitive[10:]
                                branch what

I think theres some missing (test) files from the patch that didnt get into the changeset
one at least is Tests\100-basics\042-int-minValue.cobra
But the patch code given needs a CobraCore.noOp line added anyway so it becomes

Index: Tests/100-basics/042-int-minValue.cobra
===================================================================
--- Tests/100-basics/042-int-minValue.cobra     (revision 0)
+++ Tests/100-basics/042-int-minValue.cobra     (revision 0)
@@ -0,0 +1,15 @@
+#.require. clr
+@platform clr
+namespace Test
+
+       class Test
+
+               def main
+                       is shared
+
+                       # minValue and maxValue
+                       x = 2_147_483_647
+                       x = -2_147_483_647  # to-do: should end in 8
+                       CobraCore.noOp(x)
+                       assert int.minValue <= -2_147_483_647  # to-do: should end in 8
+                       assert int.maxValue >= 2_147_483_647

I'll append a list if theres more.

[hopscc]

Looks like the changeset is fine ( no missing files).

The only thing is I missed some files that should have been changed and in the patch (:-(.
Its only java build stuff though so only affects me (:-).

Index: Source/Cobra.Lang/Java/mkjarAll
===================================================================
--- Source/Cobra.Lang/Java/mkjarAll     (revision 2637)
+++ Source/Cobra.Lang/Java/mkjarAll     (working copy)
@@ -6,7 +6,8 @@
 echo 'making RTL from java sources'
 [ -d classes ] || mkdir classes
 [ -d classes/cobra/lang ] && rm -rf classes/cobra/lang/*
-javac -d classes SourceSite.java CobraImp.java CobraCore.java Delegate.java AssertException.java CobraDirectString.java
+EXCEPTIONS="AssertException.java InvariantException.java RequireException.java
EnsureException.java NonNilCastException.java DynamicOperationException.java"
+javac -d classes SourceSite.java CobraImp.java CobraCore.java Delegate.java ${EXCEPTIONS} CobraDirectString.java
 [ $? == 0 ] || exit
 jar cvf CobraLang.jar -C classes .
 [ $? == 0 ] || exit
Index: Source/Cobra.Lang/Java/mkjar
===================================================================
--- Source/Cobra.Lang/Java/mkjar        (revision 2637)
+++ Source/Cobra.Lang/Java/mkjar        (working copy)
@@ -5,7 +5,8 @@

 [ -d classes ] || mkdir classes
 [ -d classes/cobra/lang ] && rm -rf classes/cobra/lang/*
-javac -d classes SourceSite.java CobraImp.java CobraCore.java Delegate.java AssertException.java CobraDirectString.java
+EXCEPTIONS="AssertException.java InvariantException.java RequireException.java
EnsureException.java NonNilCastException.java DynamicOperationException.java"
+javac -d classes SourceSite.java CobraImp.java CobraCore.java Delegate.java ${EXCEPTIONS} CobraDirectString.java
 [ $? == 0 ] || exit
 jar cvf CobraLang.jar -C classes .
 [ $? == 0 ] || exit

[Charles]

bump

[Charles]

I have applied your changes in changeset:2640, changeset:2641 and changeset:2642. Thanks.

[hopscc]

Mondo patch - lotta new and fixed stuff

Codegen and RTL support for Collection Literals

Support multiple classfiles (in cobra source) to make individual java
source files so javac not snarl at us.

Change cobra java build to generate a target executable jar file (and run it)
rather than a bunch of class files in cwd.
Side effects: classes placed in cobraClasses subdir in cwd

java source files still gen in cwd.

Support foreach in java codegen, etc

Base cobra Exception on java RuntimeException and support codegen for
exception catching and throwing in cobra code.
Still need to do something for java libs generating non RuntimeExceptions

Java support for ((simple) .Net like) enums.
Movement of some code for this to backend(s) - getEnumeratorMember{,Type}*

Allow portable Enum inclusion testing using 'in'
Multiple (non colliding bitset values in .Net - Ctor syntax for setting )
done using internal remap to java EnumSet for java BE.
This precludes mixing individual Enums and Enum bitsets in one
variable (for Java BE)
( need to make an example.... Should migrate to 'enumval in EnumSet'
syntax for testing inclusion anyway)

Backend handing: base and full exe filenames

KeyValuePair as a typeprovider cobra Type rather than wiring to .Net representation.

Pull BackEnd out of Compiler into own file BackEnd.cobra

Compiler:

Better debugging for missing names in containers and namespaces
(on a Development machine) - shows up when using wrong library names for a
backend (internally to the compiler)

Add support for removing name/decls from a container for
removing an existing duplicate name.

Support print redirect syntax (java)

move testing of Type for this to BackEnd

Better handling/lookup for Indexors

For Java classes symbol acquisition prioritise so that a dup of a field name
vs a method name keeps the method (rather than whatever it finds first).

Cleanup a whole lot of comments and docStrings

Platform fixups for testifyRunner

Mondo Additions to java BE RTL. - new java classes

platform specific mods and additions to testfiles

The net effect is that now a java backend testify run of Tests/100-basics
succeeds on 77 tests with only 3 failures.....

[Charles]

Looks great. Thanks for the hard work. changeset:2808

[hopscc]

Fixup to missing files so Java RTL builds.
Cleanup to remove dup info.

[Charles]

Patch java-jvm-7a.patch applied in changeset:2845

[hopscc]

Fix for broken java codegen - not filling java files with generated code since about changeset:2934.
Correction for incorrect int casts.
Some new breakage yet unaddressed from checking param vs args.
Probably from CallExpr bindImp uncommenting 'not yet ready code'

[Charles]

Patch applied in changeset:3004. I had some conflicts probably due to some renames. Check it out.

[hopscc]

Looks OK (a java build smoke test passes which was not the case before).

The java code to emit a warning on use of optional params is commented out so in java optional params would be ignored silently
- was that by design or something from one of the collisions ?