Author Topic: Using #ifdef and calling tree parsing problem?  (Read 5238 times)

Mustang

  • Community Member
  • Posts: 8
  • Hero Points: 1
Using #ifdef and calling tree parsing problem?
« on: May 16, 2008, 03:20:24 pm »
Hi!

  I've been using SE off/on for over 10 years.  However, I'm far from a power user.  I use SE mostly to understand inherited code.  Which brings up my question. :)

I'm currently using SE to determine the calling trees and references to functions. 
However, I found that SE did not properly handle a #DEFINE that was never defined.  Huh?  I had to do:
o Tools -> Options -> Languages -> Application Languages -> C/C++ -> C/C++ Preprocessing
o New ENG_TEST_BUILD
o click on ENG_TEST_BUILD
oo check the undef box.

  Then, the calling tree properly parsed out.  Is this the correct operation of SE? 

Here is the code in question:
void Maintenance(void)
#ifdef ENG_TEST_BUILD
{ ; }
#else
{
  static int local1_int1=0;
  static int local2_int1=0;
... 


BTW:
I'm using the following version of SE.
SlickEdit Version 13.0.0.0
Build Date: March 26, 2008
Emulation: CodeWright


  I did do a search first.  That's how I found out about the C/C++ Preprocessing option. :) So, I hope that my above description may also help someone else.

Thanks In Advance!
_______
Joe

chrisant

  • Senior Community Member
  • Posts: 1410
  • Hero Points: 131
Re: Using #ifdef and calling tree parsing problem?
« Reply #1 on: May 16, 2008, 04:19:35 pm »
I don't know for sure what decision SE is making under the covers, but I know that there is no right answer in the case you described:

1.  Suppose SE never assumes any preprocessor symbols are true.  Then the millions of lines of #ifdef WIN32 code will be ignored.  And typically I see code bases with #ifdef FEATURE_WE_WERENT_SURE_WAS_READY but we decided the feature was indeed ready, but never removed the #ifdef's, and all such code would be ignored, too.  And so forth.  It is very common to have "real" code nested inside #ifdef's where SE wouldn't know the conditional symbol ahead of time.

2.  Suppose SE always assumes unknown preprocessor symbols are true.  Then your type of case doesn't work.

Between the two scenarios, I think it's more important to address #1, and I suspect that's exactly what SE is doing under the covers.  It's unfortunate that #2 is a casualty, but I can't think of how to rationalize the two diametrically opposed scenarios.  Can you?  (Honest question!)

Mustang

  • Community Member
  • Posts: 8
  • Hero Points: 1
Re: Using #ifdef and calling tree parsing problem?
« Reply #2 on: May 16, 2008, 05:39:31 pm »
chrisant:

  Thanks for the reply!

  But, I'm not sure that I follow.  I have my ".h" files included in the project.  And, SE sees and parses those files fine.  I see the function definitions in my reference list. So, I think that it's "reasonable" to assume that SE would use the DEFINES listed in the project while doing it's parsing.

  From what I see, SE has many predefined DEFINES, so that most code will be parsed correctly.

  From my point of view, I don't see a gray area.  Unless something is defined, it fails an #ifdef.  That's how C compilers work.  So, that's how I would expect SE parsing to work. :)

  Actually, what *I* think would be best :) is if SE ignored the #ifdef preprocessor commands completely when doing a calling tree or reference list.  Or, at least have that option.

  If you're changing a function that you inherited,  then you want to have decent way to get a feeling on the impact of that function on other functions.  Knowing that callers and the calling tree of a function helps the person at least get "some sort" of check on the scope of that function's effects.

  The problem with having SE consider the #idef statements in doing the callers and calling tree is that both of those can change if a DEFINE is changed.  In my case, we have a number of customers and therefore there are a number of different options.  Using #ifdef commands helps in the speed and reducing the required code size.  But now, that means that I can't get a full list of all of the possible callers and functions in the calling tree.

Thanks again for the reply and any insight!
_______
Joe

chrisant

  • Senior Community Member
  • Posts: 1410
  • Hero Points: 131
Re: Using #ifdef and calling tree parsing problem?
« Reply #3 on: May 16, 2008, 07:24:20 pm »
I have my ".h" files included in the project.  And, SE sees and parses those files fine.  I see the function definitions in my reference list. So, I think that it's "reasonable" to assume that SE would use the DEFINES listed in the project while doing it's parsing.
No, SE does not use those DEFINEs while doing its parsing.  For one thing, a #define will get tagged if it is encounted in any file.  But that does not mean that it is defined in all files, or even in certain ranges of lines even in the file in which it was #define'd.  So it is not reasonable to just look up the tagged symbols.  Contextual knowledge is required, which would be achieved by doing a full recursive parse of all headers when tagging each file.
And for performance reasons SE does not use do that full recursive parse (read the text in the box at the bottom of the Languages->Application Languages->C/C++->C/C++ Preprocessing node in the Tools->Options dialog).


From what I see, SE has many predefined DEFINES, so that most code will be parsed correctly.

  From my point of view, I don't see a gray area.  Unless something is defined, it fails an #ifdef.  That's how C compilers work.  So, that's how I would expect SE parsing to work. :)
Perhaps SE does work that way, as I said, I don't know (i.e. perhaps you encountered a weird bug, not a "working as intended" thing).  But given the above point about performance considerations, I'm not sure that the right thing is for SE to work how C compilers work.  That could make things very pedantic and painstaking for most code bases to get SE to catch most stuff properly, versus making things pedantic and painstaking for a small set of stuff in a small set of code bases.  It could be that a tradeoff decision was made, to get the 90% case to work great right out of the box.


  Actually, what *I* think would be best :) is if SE ignored the #ifdef preprocessor commands completely when doing a calling tree or reference list.  Or, at least have that option.
That leads to other kinds of parsing problems.  For example, the snippet below.  Of course people can structure their code to avoid such problems, but SE can't very well tell people "use our product, but first you have to rewrite your source code so our product can understand your code".  SE has to be smarter than that, but it can't magically get things right 100% of the time, and it shouldn't force users to teach it hints 100% of the time, either.  It seems reasonable to strike a balance that involves 90% of the time magically getting it right, and 10% of the time needing hints from the user.
Code: [Select]
#ifdef DEBUG
enum SomeType {
    someDebugValue,
#else
enum SomeType {
#endif
    someRealValue,
    someMAX
};


  If you're changing a function that you inherited,  then you want to have decent way to get a feeling on the impact of that function on other functions.  Knowing that callers and the calling tree of a function helps the person at least get "some sort" of check on the scope of that function's effects.

  The problem with having SE consider the #idef statements in doing the callers and calling tree is that both of those can change if a DEFINE is changed.  In my case, we have a number of customers and therefore there are a number of different options.  Using #ifdef commands helps in the speed and reducing the required code size.  But now, that means that I can't get a full list of all of the possible callers and functions in the calling tree.
Sure, I understand the scenario.  It's just tricky, and there are multiple factors to consider.  I don't relish the thought of SE taking an extra 4 minutes per file when tagging just because it's parsing through 87MB of #include files to once in a while make a difference because it picked up two extra #defines perfectly correctly.  (I don't know how much extra time it would be, it's an exaggeration, but I really don't want it to do that parsing, I value speed more than 100% perfect accuracy, and I think most users tend to want that too, although most of us will once in a while want accuracy, and we can get accuracy by providing hints).

Mustang

  • Community Member
  • Posts: 8
  • Hero Points: 1
Re: Using #ifdef and calling tree parsing problem?
« Reply #4 on: May 16, 2008, 08:04:43 pm »
chrisant:
  Thanks for the insight!!

  I agree about the performance aspect considerations of SE!  That's why, for my main editing tasks, I avoided using SE, or other "smart" editors (like Multiedit, etc), for a while.  They were too slow for my tastes in scrolling, editing, and so on.  Now, with very fast dual processors, SE seems to have a very good response time while editing.

  I agree that this may not be the "typical case" for a lot of "GUI-based" applications.  But, for embedded, I'm seeing it more and more where the same hardware platform is used in many many different applications and products.  That's what I've been given (I'm a consultant - 15+ years of experience).

  This code/product uses a lot of #ifdef statements to do conditional compilation!  As I said, I'd have to 100% agree that it makes sense for this product.


On soapbox: :)
BTW:  Can someone please buy many of the companies out there a clue! :)  As a consultant, they pay me for each hour, and I'm not that cheap. :)  Yet, foolish bean counters" "don't want to pay" ~$300 for a SlickEdit license.  Huh?  Honestly, that's one of the stupidest things I can think of! The company is paying me how much per hour?  Yet, they will have me waste days/weeks/months because they don't want to pay $300 for an editor "because editors like EMACS are free".  IMHO, that's just plain stupid! :)

Of course, thankfully, my current company said "If you can get stuff done quicker, then no problem! :)  My personal copy is from ~2000.  So, before, I've brought code home before and used that.  But, in many cases, code can't leave the company or the secure network.

Oh yea, one more.  The above is also true for paying for better compilers. :)
Off soapbox: :) :)


Thanks again for the insight!  *I* haven't hit code before with this type of extensive use of #ifdef statements.  Thankfully, SE does have a "means" (as I mentioned initially) to "kinda" deal with this type of code.
____
Joe
« Last Edit: May 16, 2008, 08:08:49 pm by Mustang »

Dennis

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2834
  • Hero Points: 429
Re: Using #ifdef and calling tree parsing problem?
« Reply #5 on: May 27, 2008, 04:37:33 pm »
I can explain what SlickEdit really does (and some of why).

Code: [Select]
#ifdef SOMETHING
   path1
#else
   path2
#endif

If you have configured "SOMETHING" in the C/C++ preprocessing, then SlickEdit knows about the symbol and that is that, it will chose the appropriate path.

If not then SlickEdit's tagging engine parses both paths. (note that a lot of other features know nothing about the C/C++ preprocessing).  This is fine sometimes, when the result is still syntactic, but as in the example chrisant gave with SomeType, you would wind up with mismatched braces.  Another typical example, a little less harmful, is with function prototypes.

Code: [Select]
void myFunction(
                        int handle,
#if SOMETHING
                        int x, int y, int z,
                        int tm
#else
                        Point p,
                        int tm
#endif
                        );

In reality, this function has either 3 or 5 parameters, but SlickEdit's going to think it has 7.  That's because SlickEdit is even more powerful than your code.  :-)  It likes a challenge.

SlickEdit does not look at your project tag file to resolve preprocessing.  One reason is performance, another is accuracy.  We do not tag "#undef" as a symbol definition (because it, well, isn't), so for one, we wouldn't know when a symbol is explicitely undefined, and we can't know when a symbol is implicitly undefined, unless we do full preprocessing of everything including system header files.  The third reason is that sometimes your project defines preprocessor symbols different ways in different header files or preprocessing paths.  Again, it this scenario, we would be back to just guessing.  Our preference is to not guess rather than to guess wrong.

We are looking into adding a workspace level preprocessing configuration step that will let you pull in #defines from your workspace.  This too is tricky, because there are four kinds of #defines:  (a) constants, expressions, code snippets (b) code generators / hacky old-style templates (c) targets for #ifndef / #ifdef, and (d) symbols used for any/all of the above.  We are only interested in preprocessing away (b) and (c).  If we fully preprocessed all symbols, then some of your tagging results become quite boring (lose information).

Code: [Select]
#define THE_INTERNET_ON_ITS_SIDE 333
int someFunction(int arg1=THE_INTERNET_ON_ITS_SIDE);

would be tagged as

Code: [Select]
int someFunction(int arg1=333);
#if/#ifdef are just a small part of preprocessing.  There is also code-generating macros, and in fact, from a pure parsing perspective, any identifier could be a macro.  This is again why, for performance we stick to the C/C++ preprocessing setup and do not query the workspace tag file for every symbol we see.  Example:

Code: [Select]
int f _PROTO(int x, int y, int z);

If we know about _PROTO(), then no problem.  If not, this just looks like garbage (in fact, SlickEdit will tag it as a prototype for a function named _PROTO().  This is a pretty common problem, and people wonder why they don't get any tagging for their functions and why the Preview window has such a hard time listing all the definitions of _PROTO().  Defining that one macro for example, in C/C++ preprocessing can completely transform your SlickEdit experience.

But, returning to the automation question, if you have a _PROTO, chances are it's defined something like this.

Code: [Select]
#ifdef USING_ANSI
#define _PROTO(p)   (p);
#else
#define _PROTO(p)   ();
#endif

So we only know which version to use if we know what USING_ANSI is, which will ultimately depend on what compiler you are using today.

So, yeah, it's a hard problem.

hs2

  • Senior Community Member
  • Posts: 2738
  • Hero Points: 288
Re: Using #ifdef and calling tree parsing problem?
« Reply #6 on: May 27, 2008, 08:29:51 pm »
In addition #defines given as compiler flags (e.g. /DNDEBUG or -DDEBUG=1) might require some hinting in the CPP setup as well.
Quote
workspace level preprocessing configuration
Sounds great that this long standing feature request will be implemented in the near (?) future !
Would be really helpful for multi-platform development...
HS2

Mustang

  • Community Member
  • Posts: 8
  • Hero Points: 1
Re: Using #ifdef and calling tree parsing problem?
« Reply #7 on: May 28, 2008, 08:43:58 pm »
Dennis and hs2,

  Thanks for all of the help and insight.  I greatly appreciate it!
___
Joe

chrisant

  • Senior Community Member
  • Posts: 1410
  • Hero Points: 131
Re: Using #ifdef and calling tree parsing problem?
« Reply #8 on: May 28, 2008, 09:54:46 pm »
@Dennis (HP++), thanks for the clarity and detail, that is an excellent post on how Slick evaluates preprocessor directives!