Author Topic: 22.0.1 not reaping defunct processes?  (Read 493 times)

rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
22.0.1 not reaping defunct processes?
« on: February 13, 2018, 10:37:23 pm »
Currently in my ps output, I see many child zombie processes of vs_exe:

Code: [Select]
$ ps -ef | grep 2491
rbresali  2407  2491  0 Feb12 pts/12   00:01:02 [vs_exe] <defunct>
rbresali  2491  2480  3 Feb12 pts/12   01:02:42 /home/rbresali/pen/slickedit/se_latest_64/bin/vs_exe -st 0 +new -sc /home/rbresali/pen/slickedit/config
rbresali  9864 10787  0 17:35 pts/16   00:00:00 grep 2491
rbresali 18133  2491  0 Feb12 ?        00:00:00 [ssh] <defunct>
rbresali 19858  2491  0 Feb12 ?        00:00:00 [ssh] <defunct>
rbresali 20836  2491  0 Feb12 pts/12   00:00:09 [vs_exe] <defunct>
rbresali 20881  2491  0 Feb12 pts/12   00:01:04 [vs_exe] <defunct>
rbresali 28870  2491  0 11:04 ?        00:00:00 [ssh] <defunct>

For my builds, I do run a script that sshes into another machine and runs a build, but I'm pretty sure it ends normally.

These zombie processes are eating up memory.

Is this a known issue? Anything I can do to debug more?

rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
Re: 22.0.1 not reaping defunct processes?
« Reply #1 on: February 14, 2018, 03:58:47 am »
With some more thinking, I think the [ssh] zombies are because I was using FTP tool window to sftp to an ssh server. Sometimes that ssh server gets rebooted while I did not drop the connection in the FTP tool window, maybe that is the issue?

Not sure what the [vs_exe] zombies are though, but I have noticed that SE sometimes spawns other vs_exe.

Anything like this fixed in 22.0.2.1?

Also, in some other threads I saw a mention of 22.0.2.2, is that also coming out soon?

Dennis

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 2342
  • Hero Points: 308
Re: 22.0.1 not reaping defunct processes?
« Reply #2 on: February 14, 2018, 03:19:32 pm »
Yes, on 22.0.2 -- https://community.slickedit.com/index.php/topic,15996.0.html

Could you attach to one of the defunct vs_exe processes with GDB and do a "bt all" ?  I would like to see what those processes are waiting on.

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 862
  • Hero Points: 55
Re: 22.0.1 not reaping defunct processes?
« Reply #3 on: February 14, 2018, 03:22:06 pm »
Do you know yet if it happens regularly?

There aren't any known problems with waiting for child processes. I'm going to take a look at the process handling code, maybe there's some error condition where we can skip out on the wait calls and leave zombies.  No debug to collect yet; if we do end up needing to trace something, I'll need to work out something that doesn't drown your system in logs while waiting for it to happen.

Does /proc/20836/cmdline have anything in it?

rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
Re: 22.0.1 not reaping defunct processes?
« Reply #4 on: February 14, 2018, 03:37:57 pm »
It does seem to happen regularly, I've seen it a bunch of times in last few weeks.

gdb says that vs_exe is a zombie when I try to attach:

Attaching to process 20836
warning: process 20836 is a zombie - the process has already terminated
ptrace: Operation not permitted.
« Last Edit: February 14, 2018, 05:26:53 pm by rowbearto »

rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
Re: 22.0.1 not reaping defunct processes?
« Reply #5 on: February 14, 2018, 03:38:50 pm »
/proc/20836/cmdline is empty.

Marcel

  • Senior Community Member
  • Posts: 166
  • Hero Points: 13
Re: 22.0.1 not reaping defunct processes?
« Reply #6 on: February 14, 2018, 06:19:42 pm »
Defunct processes don't occupy any resources, with the exception of a process table entry.  They exist because the parent process hasn't [yet] reaped the exit status.  They will disappear once the parent goes away, or retrieves the status. 'ps' will tell you who the parent process is (appears obvious in this case). Unless you have a ton of them, it is probably not worth losing sleep over.

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 862
  • Hero Points: 55
Re: 22.0.1 not reaping defunct processes?
« Reply #7 on: February 14, 2018, 08:10:58 pm »
Looking at it, the zombie ssh process is very likely fallout from the sftp connection dropping, as you suspected. I can probably hotfix it once I verify for sure the problem with the error handling.

Not sure about the vs_exe zombie yet, that's next.

rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
Re: 22.0.1 not reaping defunct processes?
« Reply #8 on: February 14, 2018, 08:15:44 pm »
Thanks for looking.

For vs_exe, I think it is spawned when I perform multi file searches. If I stop the multi-file search, maybe SE can't handle reaping the children?

My ssh connection often drops because I am connecting to a circuit pack that reboots after I update its software.

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 862
  • Hero Points: 55
Re: 22.0.1 not reaping defunct processes?
« Reply #9 on: February 15, 2018, 02:47:23 pm »
Curiously enough, it looks like the ssh zombie is fixed by 22.0.2. There was a problem in the session cleanup code; technically you could get a zombie even on a normal connection close.


rowbearto

  • Senior Community Member
  • Posts: 938
  • Hero Points: 43
Re: 22.0.1 not reaping defunct processes?
« Reply #10 on: February 15, 2018, 03:45:53 pm »
On 22.0.2 now, I do have a zombie vs_exe:

Code: [Select]
$ ps -ef | grep vs_exe
rbresali 16432 21359  0 09:52 pts/0    00:00:00 [vs_exe] <defunct>
rbresali 21359 21349 10 09:09 pts/0    00:09:55 /home/rbresali/pen/slickedit/se_latest_64/bin/vs_exe -st 0 +new -sc /home/rbresali/pen/slickedit/config

I have not used ftp tool window yet so can't judge for the ssh issue.

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 862
  • Hero Points: 55
Re: 22.0.1 not reaping defunct processes?
« Reply #11 on: February 15, 2018, 03:49:37 pm »
Yes, we can reproduce the vs_exe zombie when we sto multi-file search in 22.0.2. 

patrick

  • SlickEdit Team Member
  • Senior Community Member
  • *
  • Posts: 862
  • Hero Points: 55
Re: 22.0.1 not reaping defunct processes?
« Reply #12 on: February 15, 2018, 05:53:24 pm »
The vs_exe multifile search zombie is not hot-fixable; the fix will be in the next release.