• Jiri Slaby's avatar
    TTY: pty, fix pty counting · 24d406a6
    Jiri Slaby authored
    tty_operations->remove is normally called like:
    queue_release_one_tty
     ->tty_shutdown
       ->tty_driver_remove_tty
         ->tty_operations->remove
    
    However tty_shutdown() is called from queue_release_one_tty() only if
    tty_operations->shutdown is NULL. But for pty, it is not.
    pty_unix98_shutdown() is used there as ->shutdown.
    
    So tty_operations->remove of pty (i.e. pty_unix98_remove()) is never
    called. This results in invalid pty_count. I.e. what can be seen in
    /proc/sys/kernel/pty/nr.
    
    I see this was already reported at:
      https://lkml.org/lkml/2009/11/5/370
    But it was not fixed since then.
    
    This patch is kind of a hackish way. The problem lies in ->install. We
    allocate there another tty (so-called tty->link). So ->install is
    called once, but ->remove twice, for both tty and tty->link. The fix
    here is to count both tty and tty->link and divide the count by 2 for
    user.
    
    And to have ->remove called, let's make tty_driver_remove_tty() global
    and call that from pty_unix98_shutdown() (tty_operations->shutdown).
    
    While at it, let's document that when ->shutdown is defined,
    tty_shutdown() is not called.
    Signed-off-by: default avatarJiri Slaby <jslaby@suse.cz>
    Cc: Alan Cox <alan@linux.intel.com>
    Cc: "H. Peter Anvin" <hpa@zytor.com>
    Cc: stable <stable@kernel.org>
    Signed-off-by: default avatarGreg Kroah-Hartman <gregkh@suse.de>
    24d406a6
pty.c 19.6 KB