Why isn't project-compile using my class path?

[crygaard]

I recently installed SlickEdit 26 on a new computer, and I'm calling the project-compile macro (it' usually invoked by pressing shift-F10, but I've remapped it so I am invoking it with shift-F6), and it is not using the classpath I have set up in Build-> Java Options, in the Classpath tab.  It uses C:\Users\username\AppData\Local\Temp\se_classpath24468.jar;root_of_project_directory for the class path.  The file C:\Users\username\AppData\Local\Temp\se_classpath24468.jar does not seem to exist, and I don't need the project root directory in my class path.

Java Live Errors is working fine, so it is clearly using the class path I specified.

I loaded Ubuntu and SlickEdit 26 onto a different partition on the same machine, and I have basically the same problem.  It is using  /tmp/se_classpath2693.jar:root_of_project_directory for the class path, and /tmp/se_classpath2693.jar does not appear to exist.  Java Live Errors works fine on this installation, also.

I have SlickEdit 21 running on another computer, and it is using the classpath as expected.

What am I doing wrong?

Chris

Update:  In an attempt to understand what is happening, I updated my SlickEdit 21 installation to SlickEdit 26.  Now it's broken in the same way.

[patrick]

I think this changed in v24.  For long classpaths that are approaching the OS limits for command line length, we now put the classpath into a manifest of a jar file, and pass the jar file path on the command line instead.   Since your build was working before, you hadn't hit the hard command line length limit, but we're conservative about when we use the jar file workaround, because the way things are staged, more flags can be added to the command line after the classpath is built up.  So we leave some room.

The jar file is generated when you start the build, and automatically deleted when vsbuild is finished, so it's normal for it to not be there when you look for it.

It doesn't effect Java Live errors, because the classpath isn't passed on the command line to the compiler in that case, it's done though a compiler API.

I'm not sure why this isn't working for you.  While there are some limitations to how you can use jars with classpaths in the manifest, I thought that there either didn't apply for our projects, or we'd warn if we ran into them.

There's a flag that can be passed to vsbuild to make it keep the jar file so we can look at it, but it looks like setting that up through the UI isn't possible.  I'm going to write a quick script that will set that up.  I'll send that to you when it's done, with instructions on how to use it, and what to look at in the classpath jar file.

[patrick]

Ok, download the attached classpathcap.e file.  Doesn't hurt to put it in a directory with a short path.  If you're seeing the same problem on Linux and Windows, I'd recommend doing this with the Linux system, since in most cases it will make the relative paths you need to look at shorter. 

With your java project open, go to the SlickEdit command line, and type in "xcom /path/to/classpathcap.e" and press return.  This will execute the script.  You should get a dialog box that says "jarfile generated at: JAR_PATH".   

You can unpack the jar file at JAR_PATH, and the class path will be in the META-INF\Manifest.txt.

Take a look at that classpath in the manifest, and see if you can see either a missing classpath, or a incorrect path.  A limitation of the manifest is that the paths must be relative to the location of the classpath jar, which for some project layouts can make the paths ugly.  Let me know what you what you find - for the missing jars or jars with bad paths, it would be helpful to know the path listed in the manifest and the absolute paths of those files.  (if there are several, just get the information for a 2 or 3 of the paths).

[Rodney]

@crygaard
Please correct the email address in your profile. You will not receive topic notification emails until you do so.

++rodney

[rygaard]

On my Windows machine, with my project loaded, I ran the script.  It did not display exactly what you had described.  Instead it displayed

jarfile generated at: project_directory\se_classpath15096.jar;class_path_directory_1;class_path_directory_2;class_path_directory_3;...

Some notes:

• The class path directories were from the class path I configured
• I did not see any directories from the %(CLASSPATH) entry in my configuration (see item 4, below)
  
• The class path directories were not in the order I configured (this is not a problem for my particular project)
• Not all of the class path directories I configured were listed.  I could not tell if this was because the dialog box was too small to show the entire path (there is no way to resize it), or if this is because some of the directories I specified were being ignored.
• The file project_directory\se_classpath15096.jar did not exist after I ran the script.

Update:

I tried running the project-compile macro (using the keyboard shortcut), and now it is using the classpath shown by classpathcap.e, instead of the shorter classpath I described in my earlier post.

Re-ordering the classpath is not a problem in my project, but it can be a serious bug in some situations.

[rygaard]

Update:

I tried running xcom /path/to/classpathcap.e from my Linux partition, and got different behavior.  The message box said:

   jarfile generated at /tmp/se_classpath2799.jar

and it did not list anything that looked like a classpath.  After running the script, /tmp/se_classpath2799.jar did not exist.

I tried running classpathcap.e in the Slick-C debugger, but it never stopped at any of the breakpoints I set.

[rygaard]

From https://docs.oracle.com/javase/8/docs/technotes/guides/jar/jar.html#classpath:

Currently, the URLs must be relative to the code base of the JAR file for security reasons. Thus, remote optional packages will originate from the same code base as the application.

Based on this, I updated my system CLASSPATH environment variable to use only relative paths (with some exceptions; see below), and I updated all of the Classpath entries in my SlickEdit configuration to use only relative paths (with some exceptions; see below).  This did not fix the problem, but it does point out a possible issue with SlickEdit.  When I go to Build->Java Options, then select the Classpath tab, then click on the Add Path... button, it inserts absolute paths, not relative paths.

On Windows, I am not aware of how to make all of my class path entries relative.  A few of my class path entries point to directories on a different drive, so I had to use absolute paths for those entries.  When I re-run classpathcap.e, the displayed class path has these entries explicitly listed, as if SlickEdit is trying to leave those entries out of the JAR-based classpath (I can't be sure because I can't get SlickEdit to work).

Any help with this would be most appreciated.

[patrick]

I haven't gotten to this yet today, but some initial comments from skimming over your responses:

1) For the windows system, that dialog probably is truncating it, if there are jars on multiple drives.   The jar classpaths need to be relative, so when we build the jar, we put everything on the same drive as the jar in the jar file, and other paths just get appended to the classpath that gets sent on the command line.  To see the un-truncated version, you can open up classpathcap.e, go to line 45 and replace

Code: [Select]

            _message_box("jarfile generated at: "actual_classpath);

with

Code: [Select]

            say("jarfile generated at: "actual_classpath);

and then run it again.  It will pop up that message in a debug window, and then you can copy it out of there with Ctrl-C and paste it into a buffer so it's easier to wrap if it's a really long line.

2) I'm not sure for the linux one yet.  The code that deletes the jar file can't be called when running that script, so the jar file must be not being created; but I need to go through the code and see how that can be possible without surfacing an error anywhere.  If you run it again on the linux box, and go to the Output tool window, are there any messages?   

3) That is correct, we have to ensure the classpath entries are relative paths when putting them into the manifest of a jar.  But that should not require you to change the classpaths you configure, our code should be making them in to relative paths.  If you see any absolute paths in the MANIFEST.MF of the jar file, then we have a bug I need to track down.

[patrick]

For the linux case, with no jar file created, I've gone through the code.  I found one place where it might fail to create the jar file and not show an error message.  It's a mistake to not show an error to the user, I'm going to fix that. 

If the problem is happening here,  the jar call to the jar command is failing, either because jar returns an error code, or because we couldn't find the jar executable.  If that's what's happening to you, there should be some error output in the Output tool window when you build.   Take a look there, and if it doesn't make any sense, send the output to me, and I'll take a look at it.

[rygaard]

Patrick,

This is where we say "Hooray!  We're making progress!  We have a different error message!"   

Based on your latest post in this thread, I looked at the "Output" tab (I had been looking at the "Build" tab).  When I try to invoke project-compile (via the keyboard shortcut), this is what I found in the "Output" tab:

EXEC jar.exe cfm "C:\Users\loginname\AppData\Local\Temp\se_classpath1852.jar" "C:\Users\loginname\AppData\Local\Temp\manifest1852.txt"
java.io.IOException: invalid header field (line 1)
   at java.base/java.util.jar.Attributes.read(Attributes.java:411)
   at java.base/java.util.jar.Manifest.read(Manifest.java:290)
   at java.base/java.util.jar.Manifest.<init>(Manifest.java:100)
   at java.base/java.util.jar.Manifest.<init>(Manifest.java:76)
   at jdk.jartool/sun.tools.jar.Main.run(Main.java:268)
   at jdk.jartool/sun.tools.jar.Main.main(Main.java:1665)

On another topic, I tried modifying classpathcap.e to use say(...) instead of _message_box(...), and it produces no output that I can find.

[patrick]

Ahh, that is interesting.  The evidence is gone, the manifest we're trying to jar up is deleted at this point.  I've attached a version of projconv.e for v26.0.2 that does not delete the manifest file.  Download that, and then load it into SlickEdit by typing into the SlickEdit command line "load \path\to\projconv.e", and then re-run the classpathcap.e script.  It should leave the manifest.mf file that's referenced in that jar error message on the Output page so you can attach it and send it to me in a reply.  And I'll take a look and see if I can spot what's wrong about what we're generating.

For the "say" changes, that should pop up a separate tool window without you having to do anything else.  It shouldn't be docked anywhere.  When you xcom the script, does any message show up in the status area where the command line appears?

[rygaard]

I don't see the attachment

[patrick]

My fault, I didn't attach it apparently.    Sorry about that.

[rygaard]

Patrick,

My fault, I didn't attach it apparently.    Sorry about that.

Been there, done that.  About a thousand times.

First:  How should I send you the file generated by projconv.e?  I don't want to post it on a public forum.

Second:  Based on the the file generated by projconv.e, I ran some experiments.  I haven't worked with the Class-Path attribute in JAR manifest files in the past, so take what I say with a grain of salt.  Anyway, I think that the format of the Class-Path attribute is invalid.  I believe the Class-Path attribute should be on one line, but the file generated by projconv.e has a separate line for each class path entry.

As an experiment, outside of SlickEdit, I created my own JAR file with only a manifest that contains the classpath, on one line.  Then I ran the javac command (again, outside of SlickEdit), and it worked just file.

As another experiment, I created my own JAR file with only a manifest, but this one contains the full, ugly classpath that I want to use for my project, again on one line.  When I ran the javac command, it said:

error: error reading path\to\my\classpath.jar; line too long (line 3)

This is a bit concerning.

[patrick]

I'll send you a PM with upload instructions in a minute. 

I'll have to look at the document for the manifest format as I look at, I was thinking you could do it in multiple lines like that, as long as you did the indent and end of lines correctly, but I also seem to remember there being some strict rules, maybe run afoul of one of those.