Author Topic: Autogenerated makefiles - absolute paths and rule issues  (Read 7854 times)

haithcoc

  • Junior Community Member
  • Posts: 4
  • Hero Points: 0
Autogenerated makefiles - absolute paths and rule issues
« on: April 15, 2011, 03:34:55 pm »
Hello,
First post on SE community forums, so I appologize if I missed something in searching the threads...
I am switching ot autogenerated/automaintained makefiles so I can improve build times by using the multiple process option of make (-j #).
In switching, however, I a few issues/questions:
1) Do %wp and %rp always include the trailing /?  The projects I have have %wp/blah/blah (vs. %wpblah/blah) in some places, and when expanded to absolute paths using _RelativeToProject in projmake.e, result in double slashes or even /blah/blah for direct children directories.
2) When %wp is used in a project (in outputfilename and other options on gnuc_options/link tab and other_options on gnuc_options/compile tab, as well as in prebuild/postbuild commands),
it gets expanded to an absolute path.  This causes issue if we want to capture the makefiles in source control.  I solved this by changing projmake.e, but is me changing a delivered macro really the best way to go.  I could create my own version, but then how do I get it to be the macro called when the makefile is generated/modified within vs (when adding files, selecting generate makefile, making other project changes).  I would like it to be as seemless as possible, and not get stepped on when vs updates are made.
3)  The current build rule for OUTFILE in the makefile always forces a link if there are dependencies, because the deps rule will always be active (cd to dependency dir and run make).  No problem with running the deps rule - the problem is that it causes LINK to be run since OUTFILE rule is dependent on OUTFILE and deps.  This effect can trickle up in a workspace with several layers of dependency, causing the same library to be built multiple times, even though it is up to date (makes run from deps all result in "Nothing to be done for all").

Thanks in advance for your help,
Stephen

Graeme

  • Senior Community Member
  • Posts: 2432
  • Hero Points: 322
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #1 on: April 18, 2011, 02:15:05 pm »
Hello,
First post on SE community forums, so I appologize if I missed something in searching the threads...
I am switching ot autogenerated/automaintained makefiles so I can improve build times by using the multiple process option of make (-j #).
In switching, however, I a few issues/questions:
1) Do %wp and %rp always include the trailing /?  The projects I have have %wp/blah/blah (vs. %wpblah/blah) in some places, and when expanded to absolute paths using _RelativeToProject in projmake.e, result in double slashes or even /blah/blah for direct children directories.
2) When %wp is used in a project (in outputfilename and other options on gnuc_options/link tab and other_options on gnuc_options/compile tab, as well as in prebuild/postbuild commands),
it gets expanded to an absolute path.  This causes issue if we want to capture the makefiles in source control.  I solved this by changing projmake.e, but is me changing a delivered macro really the best way to go.  I could create my own version, but then how do I get it to be the macro called when the makefile is generated/modified within vs (when adding files, selecting generate makefile, making other project changes).  I would like it to be as seemless as possible, and not get stepped on when vs updates are made.
3)  The current build rule for OUTFILE in the makefile always forces a link if there are dependencies, because the deps rule will always be active (cd to dependency dir and run make).  No problem with running the deps rule - the problem is that it causes LINK to be run since OUTFILE rule is dependent on OUTFILE and deps.  This effect can trickle up in a workspace with several layers of dependency, causing the same library to be built multiple times, even though it is up to date (makes run from deps all result in "Nothing to be done for all").

Thanks in advance for your help,
Stephen

Not sure if I'm following much here but for 1)  - in _parse_project_command you can see this code for processing %wp

Code: [Select]
         case 'P':
            len=3;
            s=strip_filename(workspace_filename,'N');
            break;

You could write a small macro to test what strip_filename returns for a filespec.

For 2)  - what did you change in projmake.e?  What do you mean you could create your own version?

For 3) - sorry I don't use auto-generated makefile.  Maybe you can tweak the code in projmake.e to fix this too?

Graeme


haithcoc

  • Junior Community Member
  • Posts: 4
  • Hero Points: 0
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #2 on: April 18, 2011, 04:50:57 pm »
For 1) - I did run a test, and it showed trailing slashes - I just wondered if this is always the case.

For 2) - I added a function to make absolute paths embedded in strings to relative paths.  I made the changes to projmake.e, but I am worried that these will get overwritten when updates to SE are downloaded, so I made a copy as a different filename.  I would also like to distribute this change to other members of my team, but would like it to be easy to implement.

For 3) - I made a change that works for this.

Graeme

  • Senior Community Member
  • Posts: 2432
  • Hero Points: 322
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #3 on: April 19, 2011, 11:37:56 am »
For 1) - I did run a test, and it showed trailing slashes - I just wondered if this is always the case.

For 2) - I added a function to make absolute paths embedded in strings to relative paths.  I made the changes to projmake.e, but I am worried that these will get overwritten when updates to SE are downloaded, so I made a copy as a different filename.  I would also like to distribute this change to other members of my team, but would like it to be easy to implement.

For 3) - I made a change that works for this.


1) The default behavior of strip_filename will never change regarding trailing backslashes because that would create incompatibility, likewise with %rp and %wp.

2) If your changes are useful to others, there's a chance SlickEdit would include them in the shipped code if you sent them to slickedit support and explained what they do.  There's at least one person (chrisant) who keeps all of the shipped slick C source in revision control.  He's made a large number of changes to the shipped code to get behavior that he wants at the price of having to merge his changes every once in a while.

In case you didn't know, if you make a copy of projmake.e in some folder of your own (e.g. c:/myfolder), modify it then load, it "replaces" the version of projmake.e in the slickedit macros folder  -  "replaces" means that the existing projmake.e first gets unloaded (all names defined in it get removed from the names table), then the new version in your folder (myfolder) gets loaded and all its names get added to the names table.  For example, when you load a hotfix, the associated "fixed" macro files get stored in your config/hotfix folder rather than overwriting the version in your slick installation macros folder.  If you unload a hotfix, the version in your hotfix folder gets unloaded and the original version in the slick macros folder gets reloaded.

If you make a copy of projmake.e, modify it, give it a different name (e.g. pj2.e) and load it, something slightly different happens.  The original projmake.e doesn't get unloaded when you load pj2.e but the definitions in pj2.e override the same names in projmake.e.  As you've found, this also gives you what you want but I'm guessing it's slightly better to keep the name projmake.e unchanged for your version of it  e.g. you won't have to update the slick C tag file manually and functions like _on_load_module_projmake() will still work.  (There probably is no _on_load_module_projmake currently but it's just an example of why keeping the name unchanged could be better).  One reason why you might use a different name is when you want to redefine only a small number of things in the file  e.g. slickedit lets you redefine the save_file function in a module of your own to customise what happens when a file is saved.

Also, there may be complications I don't know about involving #import when some other slick C module gets reloaded and that module #imports projmake.e but this probably isn't a problem since the hotfix mechanism works.

Every time you load a hotfix, you have to check whether any of the hotfixed files correspond to files you have modified.  If your changes are important, it's not a big price to pay for the ability to customize slick, as long as your changes aren't complicated.

Graeme

haithcoc

  • Junior Community Member
  • Posts: 4
  • Hero Points: 0
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #4 on: May 12, 2011, 09:17:34 pm »
I have a copy of the projmake.e file in another directory, and can load it and get the proper behavior (all calls to projmake.e commands use my version - this includes "Generate Makefile" from the project menu).  Now, I would like to have this loaded automatically when vs loads.  No problem for me once I load it manually, but I want others in my group to use the same projmake.  I could have them load it manually (once), but there are further complications.

We use make instead of built-in vsbuild to build our code (primarily due to multi-processor build capability).  Since they are automatically generated, we do not source control the Makefiles - just the .vpw and .prj files.  So, when we do a fresh checkout on the nightly build machine, we need to generate the makefiles before building.  This can be done by running from the terminal command line - something like:  vs +new <.vpw or .vpj> -p <script that calls generate_makefile> (as shown in post http://community.slickedit.com/index.php?topic=1133.0 using gmkf.e).  We have targets in the vpj that allow us to run this from vsbuild on the terminal command line and it generates each makefile if it is older than the vpj file, or if it doesn't exist.  That part works great.

The issue is "automatically" insuring that the modified projmake is getting used by all users (including the nightly build).  I have tried putting load(mypath/projmake.e) in the gmkf.e's defmain, but am having problems - sometimes it seems to work, and other times not.  I've tried it with load, makeNload, and xcom.  Is there a better way that will be persistent across slickedit updates?  I could have a script put something in a user.e that calls the load mypath/projmake.e everytime slickedit is run.  Any thoughts?

haithcoc

  • Junior Community Member
  • Posts: 4
  • Hero Points: 0
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #5 on: May 12, 2011, 09:21:24 pm »
I have also been experiencing some issues where sometimes -p file.e works, and sometimes it starts not working, and doesn't seem to get fixed unless I blow away parts of my config directory.  Not sure if all my attempts at getting file.e to load gets things in a confused state.  It seems that I get to where I cannot run any macro from the terminal command line (even a hellow.e).

Graeme

  • Senior Community Member
  • Posts: 2432
  • Hero Points: 322
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #6 on: May 13, 2011, 12:42:35 pm »
Quote
The issue is "automatically" insuring that the modified projmake is getting used by all users (including the nightly build).  I have tried putting load(mypath/projmake.e) in the gmkf.e's defmain, but am having problems - sometimes it seems to work, and other times not.  I've tried it with load, makeNload, and xcom.  Is there a better way that will be persistent across slickedit updates?  I could have a script put something in a user.e that calls the load mypath/projmake.e everytime slickedit is run.  Any thoughts?

Quote
I have also been experiencing some issues where sometimes -p file.e works, and sometimes it starts not working, and doesn't seem to get fixed unless I blow away parts of my config directory.  Not sure if all my attempts at getting file.e to load gets things in a confused state.  It seems that I get to where I cannot run any macro from the terminal command line (even a hellow.e).

The environment when a batch macro executes may not be suitable for doing a "load" from within the batch macro, especially a batch macro that's initiated with -p.  Without asking slickedit support, there's no way to know if this should work reliably.  Also there's the issue of the configuration (state file) maybe needing to be saved after you load a module  - hotfixMakeNload does _config_modify_flags(CFGMODIFY_LOADMACRO) to get this to happen.  You can call save_config() directly if you want.

Probably the best chance of getting something reliable (apart from asking slick support), is to copy the hotfix load mechanism  - since it's purpose is to load slickedit macro files.  Have a look at hotfixApplyModule in hotfix.e (note how it switches VSLICKINCLUDE) and hotfixMakeNload.   Also note that it calls _load rather than load.

BTW - don't know if you're doing this but you shouldn't "load" a batch macro, slick will probably get confused if you do.

So...  maybe you could try a non batch macro that firstly does the hotfixMakeNload thing, then calls the generate makefile thing, then calls exit() to close the editor.

Regarding slickedit updates -  if you're loading hotfixes or installing a version upgrade, I guess you realise you're going to need to check if projmake.e has changed from the version you based yours on and update it if so.  Maybe whoever installs the  hotfixes or does the version upgrade should also reload projmake.e.  You may be able to determine programmatically whether your version of projmake.e is loaded, maybe using find_index  - or just call a new macro function in your modified projmake.e that won't exist if your projmake.e hasn't been loaded.  Your "users" will get a slick C runtime error and hopefully notice.  Just an idea -  I know you want it to be automatic.  You could also write a non slick C program to check the time/date of all the projmake.ex files you can find and make sure yours is the newest - if not, invoke slick to do the make and load thing and save the state file (call save_config), check projmake.ex again, then run slick to generate the make file. Though... you can use slick C to check file time/dates as well.

Graeme

bunbun

  • Community Member
  • Posts: 29
  • Hero Points: 2
Re: Autogenerated makefiles - absolute paths and rule issues
« Reply #7 on: October 27, 2011, 01:55:42 pm »
Sorry to hop onto this so late.
I too use makefiles to build in parallel. I do notice that slickedit makefiles for c/c++ are not necessarily how I would write them. There are two main issues:
1) If multiple source directories are used, and both contain a source file with the same name, there is no way to specify which one will be picked up, even though only one is part of the project...
It seems to me that this is back to front.
2) header files dependencies are not included.
So, if you modify some header file, your source might not be recompiled.

Currently, I ignore (1) but modify projmake.e to add support for using "gcc -M" to generate dependency files.
Leo